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SUMMARY 


A flight simulation experiment was performed to determine 
the effectiveness of synthesized voice approach callouts for air 
transport operations. Flight deck data was first collected on 
scheduled air carrier operations to describe existing pilot-not- 
f lying callout procedures in the flight context and to document 
the types and amounts of other auditory cockpit information dur- 
ing different types of air carrier operations. A flight simula- 
tion scenario for a wide-body jet transport airline training 
simulator was developed in collaboration with a major U.S Air 
Carrier and flown by three-man crews of qualified line pilots as 
part of their normally scheduled recurrent training. Each crew 
flew half their approaches using the experimental synthesized 
voice approach callout system (SYNCALL) and the other half using 
the company Pilot-not-Flying approach callout procedures (PNF). 
Airspeed and sink rate performance was better with the SYNCALL 
system than with the PNF system for non-precision approaches. 
For the One-Engine Approach, for which SYNCALL made inappropriate 
deviation callouts, airspeed performance was worse with SYNCALL 
than with PNF. Reliability of normal altitude approach callouts 
was comparable for PNF on the line and in the simulator and for 
SYNCALL in the simulator. However, SYNCALL was more reliable 
than PNF for making deviation approach callouts in the simulator. 
Pilots generally favored the concept of SYNCALL and judged it 
more reliable than PNF callouts. They suggested modifications 
before it would be appropriate for operational use. It was con- 
cluded that SYNCALL improved flight performance for non-precision 
approaches, and that a SYNCALL system should make deviation cal- 
louts only. For consistency, it is recommended that the modes of 
the Ground Proximity Warning System (GPWS) be incorporated into a 
SYNCALL system. The detrimental effects on performance associat- 
ed with inappropriate deviation callouts led to the further con- 
clusion that such callouts should be designed out of the system. 
Finally, the results and conclusions of the experiment are used 
to develop suggestions for improvements to SYNCALL before further 
testing . 
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SYNTHESIZED VOICE APPROACH CALLOUTS FOR AIR TRANSPORT OPERATIONS 


Carol A, Simpson 


INTRODUCTION 

The final approach and landing segment of jet transport 
operations requires exact and constant awareness by the crew of 
altitude and position. This information is currently provided in 
visual form by the flight instruments and in spoken form by cal- 
louts made by whichever pilot is not actively flying the ap- 
proach, Airline procedures call for the pilot-not-flying to moni- 
tor the performance of the pilot-flying by scanning the flight 
instruments and calling out specific altitudes and any observed 
deviations outside company-prescribed tolerances. Prior to the 
point where the pilot-flying expects to be able to see the run- 
way, one pilot (depending on the airline) begins looking outside 
and reports when the runway is in sight. The other pilot focuses 
visually inside the cockpit and monitors the flight instruments. 
This pilot and, in some cases, the flight engineer are then 
responsible for making the necessary approach callouts. The 
specific set of callouts used and assignment of responsibility 
for making the callouts to pilot-not-flying and/or to the flight 
engineer varies somewhat across air carriers. Despite some 
differences in the choice of callouts, the intended functions of 
the callouts are common to all carriers. These are 1) to reduce 
the visual workload for the pilot-flying, 2) to keep all three 
crewmembers aware of aircraft altitude and position, and 3) in 
the case of callouts that require acknowledgment from the pilot- 
flying, to serve as a check for incapacitation of the pilot- 
flying . 

In the discussion that follows, the term "pilot-not-flying 
callouts" will stand for all of the approach callouts regardless 
of which crew member is actually expected to make particular cal- 
louts, This will permit discussion of the approach callouts 
themselves without referring to the differences among air car- 
riers. 

A number of air carrier approach and landing accidents during 
low or impaired visibility have been associated with the absence 
of approach callouts. This fact at first suggests that the ab- 
sence of altitude and deviation callouts may have contributed to 
inadvertent flight into terrain during these approaches. Such a 
conclusion was drawn by the National Transportation Safety Board 
in a special study on flight crew coordination procedures in ac- 
cidents during air carrier instrument landing system approaches. 
(1) However, due to the absence of data on callout performance 
for successful approaches made during the same period (1970-1975) 



no test of this suggested relationship of cause and effect can be 
performed . 

Another untestable post hoc hypothesis is that increased 
workload during the approaches that terminated in accidents 
prevented the pilot-not-flying from making the callouts. While 
neither hypothesis can be directly tested, it is reasonable to 
ask whether the current procedure can be improved, especially 
when unexpected events cause a high workload in the cockpit. One 
alternative medium for information transfer would be a speech 
synthesizer which could receive raw data from the onboard central 
air data computers and automatically make the callouts at ap- 
propriate times during the approach. 

OBJECTIVE 

The purpose of the present study was to compare a pilot-not- 
flying ( PNF ) approach callout system to a system composed of PNF 
callouts augmented by an automatic synthesized voice callout sys- 
tem. A full task flight simulation experiment was conducted to 
determine if one or the other system transfers altitude and devi- 
ation information more reliably than the other and/or results in 
better flight performance by the pil o t- f 1 ying . 

EXPERIMENTAL APPROACH 

This objective was accomplished by a collaborative effort 
between American Airlines Flight Academy and NASA Ames Man Vehi- 
cle Systems Research Division. A DC-10 flight training simulator 
was flown during normal flight training using crews as assigned 
but on a strictly voluntary basis. All pilots offered the oppor- 
tunity to participate did so. Twenty 3-nian crews flew several 
types of approaches half with the standard callout procedures and 
half with a synthesized voice callout system (SYNCALL), Measure- 
ments were made of flight performance, callout reliability, cal- 
lout interference with other audio events, and pilot ratings of 
callouts on several scales. 

The experimental approach actually incorporated two types of 
data collection: 1) Flight Deck data collection and 2) data col- 
lection during the formal flight simulator experiment comparing 
pilot-not-flying callouts to SYNCALL callouts. 
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FLIGHT DECK DATA COLLECTION 


Design of an automatic approach callout system and measure- 
ment of the effectiveness of approach callouts in the flight con- 
text in a way that is meaningful for airline flight operations 
requires some degree of familiarity with flight operations and 
the complex environment in which those operations take place. 
The author did not at the onset of the project, have the re- 
quisite familiarity with multi-engine jet transport airline 
operations. Thus the first phase of the project was a program of 
flight deck observation by the author. The necessary authoriza- 
tions from the air carrier and the Federal Aviation Administra- 
tion ( F A A ) were issued for "the purpose of observing flight crew 
activity to gain insights into the potential uses and pitfalls 
for synthesized speech." 

A systematic approach was taken for the flight deck observa- 
tion program. The program goals were: 

1) To gain over-all familiarity with airline flight opera- 
tions and the tasks performed during each phase of the flight by 
each crew member and to observe interactions among crew members, 

2) To collect data on time spent with voice communications 
with ATC as a function of phase of flight. Tnis was needed in 
order to assess the possibility of simultaneous voice messages 
from ATC and a synthesized voice callout system. 

3) To collect data on the frequency of and type of oc- 
currences of simultaneous or overlapping voice messages from mul- 
tiple sources and to observe the methods used by the crew to cope 
with such situations; 

4) To become familiar with the existing pilot-not-f lying ap- 
proach callout procedures as a function of type of weather, type 
of approach, and pilot flying; 

5) To observe any instances in which pil o t-not- f 1 ying cal- 
louts were not made to determine possible reasons for each such 
instance, e.g. distraction from ATC, higher than normal or dif- 
ferent from normal workload for the pilot-no t-flying , intentional 
omission of certain callouts due to approach circumstances, etc; 

6) To collect the above data for a cross section of type of 
flight operations - long, cross-continental, short haul, over- 
water, night, day, large or small airport, high or low traffic 
density, type of approach; 

7) To have an opportunity to observe instances of the crew 
performing other than normal operations. Since abnormal or emer- 
gency operations are extremely rare, a large number of observa- 
tion flights was needed in order to obtain any such instances. 

8) To collect crew preferences for the possible uses of syn- 
thesized speech in the cockpit while they were in the actual 
cockpit environment. 1 


1. Such conversations were conducted only during the cruise 
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Approach and Landing Procedures 
The following narrative description of crew approach and 
landing procedures is based on the author's experience and data 
collected for a sample of flights or "trip legs" during which 
scheduled air carrier operations were observed from the "jump 
seat" directly behind the captain's seat. Table 1 lists the 
types of aircraft and flight operations that were observed. The 
flight crew was observed to be a close-knit team with each member 
performing certain functions. Throughout the approach the crew 
members must use a certain amount of judgment in planning when to 
carry out their tasks so as to complement rather than interfere 
with each other. The approach callouts are only a part of the 
procedures carried out during the approach to landing and as such 
will be described in this context. 


TABLE 1. TYPES OF FLIGHT OPERATIONS OBSERVED 
NUMBER OF FLIGHTS 72 

NUMBER OF DIFFERENT AIRPORTS 24 


AIRCRAFT 

TYPES : 

TRIP 

DURATION: 


727 

30 


< 1 HR 

5 

707 

16 

1 

- 2 HRS 

21 

DC-10 

18 

2 

- 3 HRS 

13 

747 

8 

3 

- 4 HRS 

19 




> 4 HRS 

14 


TIME OF DAY: 
DAY 51 

NIGHT 21 


TYPE OF OPERATION: 

DOMESTIC PASSENGER 66 
OVERWATER PASSENGER 2 
DOMESTIC FREIGHT 4 


Throughout the entire flight the captain and the first off- 
icer are responsible for flying the aircraft with the captain 
having final decision-making authority as pilot-in-command. Typi- 
cally, the captain and first officer alternate "legs" actually 
flying the aircraft so that each pilot receives time at the 
flight controls. So, for a given approach one of the pilots will 
fly the aircraft, either manually or using one of several autopi- 
lot modes. The other pilot communicates by voice over the radio 
with the ground-based air traffic controllers. 


segment of the flight and only at the convenience of the crew and 
with the Captain's permission. Ongoing conversations were halted 
immediately, often in m id - sentence , whenever any flight duty, 
such as communication with ATC, was required. 
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The Flight engineer, in addition to his primary responsibili- 
ties as the aircraft systems operator, is usually delegated addi- 
tional monitoring and communications responsibilities. During 
Take-Off and Landing he serves as a back-up for the two pilots to 
double-check that checklist items are performed. 

The flight engineer’s back-up function is particularly impor- 
tant during the approach to landing since the pilots are busy 
with frequent changes in ATC communications and navigation fre- 
quencies, and aircraft heading, airspeed, and altitude. Table 2 
lists the observed average numbers of such changes that were 
given to the crew by ATC per descent and approach. In addition 
to complying with instructions from ATC, the pilots must make 
other changes in heading, altitude, airspeed, and aircraft confi- 
guration in order to follow their desired flight route to runway 
touchdown . 


TABLE 2. MEAN NUMBERS OF ATC CHANGES DURING 

DESCENT & APPROACH FOR 30 SCHEDULED AIR CARRIER 
FLIGHTS ACROSS FOUR AIRCRAFT TYPES AND SIXTEEN AIRPORTS 




mean 

sd 

Segment 

Time (Beginning of Descent to Landing) (min) 

2 4.0 

7.5 

Number 

of Heading Changes 

2.6 

1 .8 

Number 

of Altitude Changes 

5.3 

1.3 

Number 

of Frequency Changes 

4.1 

1 . 1 

Number 

of Speed Changes 

1.8 

1 .7 

TOTAL 

Number of Changes 

13.8 



Preparation for the approach begins while still at cruise al- 
titude. The flight engineer tunes his communications radio to 
the frequency on which the Automatic Terminal Information Service 
(ATIS) broadcast is being transmitted. He copies down the infor- 
mation on winds, runways in use, and approaches to be expected. 
He then re-copies the information onto a prepared form and passes 
it forward to the pilots. Once the pilots have this information, 
they can plan the approach. They select the appropriate approach 
charts and review the flight profile in terms of altitudes, navi- 
gational fixes, distances, headings, navigational aid frequen- 
cies, and missed approach procedures. The captain gives any spe- 
cial instructions warranted by the particular conditions. 

The approach segment, which has been defined here as starting 
when the ATC approach controller clears the aircraft for the ap- 
proach, usually starts at approximately 3000 feet Above Field 
(AFL) and includes interception of 


Level 
course , 


the final approach 
passage over a final approach fix, and descent down the 
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desired glidepath to touchdown on the runway. All of the Ap- 
proach Callouts are made during this segment, which lasts some 8 
to 10 minutes. Thus this segment was chosen for the comparative 
study of the two callout systems. 

Figure 1 is a reproduction of the page in the airline’s 
flight manual describing the approach callouts. During the 
course of the approach the pilot-flying must intercept the final 
approach course and descend to the runway touchdown point at the 
necessary rate so as to maintain the desired glidepath. Tracking 
of the glidepath may be accomplished either by reference to raw 
data such as time, speed, and rate of descent or by reference to 
a ground- transmitted glideslope signal. If the latter is avail- 
able, the approach is classified as a precision approach, and the 
approach can be flown to a relatively lower altitude and with 
less visibility than is possible without a glideslope signal. If 
a glideslope signal is not available, then that approach is 
called a non-precision approach. Either type of approach can be 
flown using what is called ’’raw data” or ufXing command data from 
a ’’Flight Director”. The data provided directly by the flight 
instruments is called ’’raw data”. The Flight Director, on the 
other hand, processes the raw data and provides maneuvering com- 
mands via steer and pitch bars or needles or similar display, 
depending on the aircraft. With a Flight Director, the pilot 
flies according to the steer and pich commands on the display. 

For either type of approach, there is a minimum altitude to 
which the aircraft is permitted to descend without having the 
runway in sight. In the United States, this is called Minimum 
Descent Altitude (MDA) for non-precision approaches and Decision 
Height (DH) for precision approaches. If the runway is in sight 
by the time the minimum altitude is reached and if the pilot- 
flying judges that the landing is assured, then the approach is 
continued to landing (or go-around , should the situation subse- 
quently become unfavorable for landing). There is a further de- 
tail in the case of non-precision approaches, those using an MDA. 
For these there is also a Missed Approach Point (MAP) at a speci- 
fied point on the final approach course, If**at MDA the runway is 
still not in sight, the aircraft can be flown at MDA altitude un- 
til the MAP is reached, descending to landing if the runway is 
acquired visually. Upon reaching the MAP with no runway in 
sight, the crew makes a go-around. 

During the approach the pilot-not-flying monitors the ap- 
proach, operates the slats, flaps, and landing gear as desired by 
the pilot-f lying , and prepares mentally to make a go-around if 
necessary. He makes the normal approach callouts applicable to 
the particular type of approach being flown, as indicated in Fig- 
ure 1. And, if deviations occur in airspeed, sink rate, localiz- 
er, or glideslope, he calls these to the attention of the pilot- 
flying. 

The Flight Engineer finishes the ’’Before Landing Checklist” 
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NORMAL PROCEDURES 


A V A 

DC-10 Operating Manual 


Section 3 


Page 63 
2-27-76 


STANDARD CALLOUTS - ALL DESCENTS, APPROACHES AND LANDINGS 


/ ^^Transition Level * 

I 1 1,000' MSI. ( EL 110 it Transition Level is at or below) 

F/O CALLOUTS 


G/S interception / 

(if making ILS approach) 

FGS Mode Annunciators 

Final Approach Fix and 
AFL Crossing Altitude 


CAPTAIN CALLOUTS 



** PILOT NOT FLYING CALLOUTS 

1,000' AFL — Verbally Verify When Flaps at Landing Setting 

V/ 

\ 500' AFL, Airspeed ± Approach Speed 

and Descent Rate 

400' AFL 
300/ AFL 


W 200' AFL 50' Radio Altimeter 
1 nu To Touchdown At 


z 10' Increments 




Pilot Not Flying 

| Airspeed - With Landing Flaps anytime speed varies from approach speed by more than ±5 Kts. 

Rate of Descent — When Below: If Descent Rate Exceeds: * 

2,000’ 2,000 FPM 

1,000’ 1,000 FPM 

300; * 700 FPM 

During Cat II Approaches F/O will make all callouts from 500' to landing, including Airspeed or Descent Rate if 
not within limits. 

LOC and GS indication Callout-On final, anytime any crewmember observes LOC displacement greater than 1/3 

dot and/or G/S displacement greater than 1 dot, other pilot will acknowledge 
NOTES* this deviation. 

1. Use the Captain’s or F/O’s altimeter for all altitude callouts, except for the CAT II approach, in which case 
use the radio altimeter for the 300' callout and remaining callouts below 300' to touchdown. 

2. When executing a non-precision approach, the pilot not flying will call out “100' ABOVE MDA,” MDA 
and MAP. 

3. Callouts not applicable on non-precision or visual approaches should be disregarded. 

* For Transition Level outside U.S. refer to avigation charts. 

F/E MONITORING 

In addition to his normal monitoring of engine instruments and F/E panel, the F/E will monitor flight 
instruments, especially altitude and airspeed. He will assist in maintaining a watch for traffic and other factors 
that could adversely affect safety. He will monitor HSI and ADI navigation indications and call out any 
discrepancies between the two instruments. On final, call out LOC displacement anytime it is greater than 1/3 
dot and/or G/S displacement anytime it is greater than 1 dot. 


Figure 1. - American Airlines DC-10 Approach Callout Procedures, 
Flight Manual, page 53, 2-27-75. Reprinted by permission. 
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to ensure that all systems are set for landing or, where ap- 
propriate, armed for go-around. He assists the Pilot-not-Flying 
with the monitoring of the approach. 

Since this entire segment lasts only some 8 to 10 minutes, it 
should be apparent that there is a fair amount of intra-cockpit 
voice communication that occurs in a short time, along with the 
ATC communications , 

ATC Communications Workload by Phase of flight 
It was found that the amount of time used by the crew to com- 
municate with ATC varied as a function of phase of flight. 
Correspondingly, so did the number of ATC transactions (consist- 
ing of one exchange between pilot and controller of either 1) pi- 
lot transmission to controller + controller acknowledgement or 
other response, or 2) controller transmission to pilot + pilot 
acknowledgement or other response). Table 3 lists this data. 


TABLE 3. AIR TRAFFIC CONTROL VOICE COMMUNICATIONS WORKLOAD 

BY PHASE OF FLIGHT. MEANS FOR 39 TRIPS. STANDARD 
DEVIATIONS IN PARENTHESES. 


PHASE OF FLIGHT 



TAXI 

CLIMB 

CRUISE 

DESCENT 

TAXI 

TOTAL 

SEGMENT 

DURATION (min) 
(S.D. ) 

19.68 

(12.33) 

24.77 

(11.48) 

86.68 

(71.55) 

25.61 

(11.72) 

5.53 

(3.66) 

163.41 
(76.41 ) 

TIME TALKING/ 
LISTENING (min) 
(S.D. ) 

1 .45 
(1.63) 

2 .83 
(1.44) 

4.30 

(3.47) 

3.90 

(1.83) 

0.47 

(0.50) 

13.58 

(5.50) 

PERCENT OF TIME 
SPENT W/ ATC 

7 % 

1 1 % 

5 % 

• '» % 

3 t 

8 % 

NO. OF ATC 
TRANSC ACTIONS 
(S.D. ) 

9.25 

(4.09) 

17.29 

(8.34) 

21.81 

(17.3D 

20.89 

(8.99) 

2.87 

(2.60) 

70.05 

(23.55) 

MEAN TIME PER 
TRANSACTION ( 3 ) 
(S.D. ) 

8.65 

(6.00) 

8.86 

(3.21) 

12.55 

(6.48) 

11.84 

(5.09) 

9.06 

(6.42) 

11.70 

(3.46) 


Overlap.pjng Aud i q Signal 3 and Voice Communications 
The types of situations in which overlapping of audio signals 
and/or speech communications occurred can be classified as either 
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routine or unexpected. That is, certain combinations of overlap- 
ping audio occur on nearly every flight, necessitating either 
formal or informal crew procedures for handling them. Others oc- 
cur occasionally by chance and must be handled by the crew on an 
event-specific and unique basis. 

Many of the routine pairs of overlapping audio signals in- 
volve concurrent voice communications between 1) pilot-not-flying 
and ATC and 2 ) Flight Engineer and AIRING (Aeronautical Radio In- 
corporated) or Company. Some examples are: 1) ATC Push-back or 
Taxi Clearance and Flight Engineer Time Out Report, 2 ) Initial 
ATC transaction with Departure Control and Flight Engineer's Time 
Off and Load Report, 3) Initial Descent from Cruise Clearance and 
F/E Monitor the ATIS, 4) Outer Marker Passage with Captain's Cal- 
lout and Pilot-not-Flying contact Tower. Except for the last 
pair, the need for concurrent audio signals is handled routinely 
by assigning the ATC communications tasks to the Pilot-not-Flying 
and the AIRINC and Company Communications tasks to the Flight En- 
gineer. However, in the case of the Outer Marker Callout by the 
Captain and the Pilo t-not-Flying ATC call to the Tower, it hap- 
pens that 50$ of the flights involve the Captain and the Pilot- 
not-Flying being one and the same individual. This individual 
must then sequence the two voice messages, with the result that 
one or the other may be late. When the First-Officer is the 
Pilot-not-Flying , he may be calling the Tower at the same time as 
the Captain is making the Outer Marker Callout, with the result 
that he may not verbally process the callout; The Captain can and 
was observed to delay the callout until after contact with the 
tower had been established (duration of approx 8 s during which 
the aircraft descends another 136 feet at, say 1000 FPM) . Thus, 
the First Officer as pilot-flying on these occasions will see an 
altimeter reading that could be as much as 100 feet different 
from that which the Captain as pilot-not-flying calls out. Types 
of overlapping auditory messages that occurred by chance included 
SELCAL or CABIN CALL bells or chimes overlapping with ATC commun- 
ications or with crew cockpit voice communications. The Gear 
Warning Horn, which was observed to trigger several times per 
flight inappropriately, sometimes overlapped ATC communications 
or crew checklist callouts. Crew members handled these situa- 
tions by sharing the workload, i.e. the Flight Engineer would 
respond to SELCALL or CABIN CALL while the pilot-not-flying would 
respond to the ATC transmission. There were instances when the 
pilot-not-flying requested a repetion of the message from the 
controller. There were also instances when the Flight Engineer 
delayed his response to the CABIN CALL until after completing his 
other verbal transaction, say reporting observed winds aloft to 
the company. 

There were occasional instances of individual approach cal- 
louts omitted when they would have overlapped with other voice 
communications. On one flight, for example, the controller ad- 
vised the crew of men and equipment beside the runway when the 
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aircraft was at 300 feet AFL. The 300-foot callout was not made 
in this instance although other hundred foot-foot callouts were 
made. 

Occasionally certain ten-foot callouts were systematically 
omitted by the PNF when the sink rate during flare was relatively 
higher. The PNF was observed to call "50, 30, 10". This seemed 
to convey the added information about the higher than average 
sink rate to the pilot-flying. 

There were no approaches observed for which no callouts were 
made. Rather, individual callouts were systematically omitted 
according to specific circumstances of the approach. But in 
these cases, prior and succeeding callouts were made. 
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SYNTHESIZER CALLOUT SYSTEM (SYNCALL) 


Figure 2 shows the experimental SYNCALL system designed to 
test the concept of automatic approach callouts. This system was 
designed to include all of the types of automatic approach cal- 
louts that might be useful. As such it was an experimental tool 
rather than a prototype system. 

System Design 

The synthesizer was programmed to make both normal and devia- 
tion callouts. The normal callouts were the standard callouts 
then required for all approaches by the participating airline. 
Table 4 lists the normal callouts made by the SYNCALL system with 
the flight condition that triggered each callout listed in the 
left column and the wording of each corresponding callout in the 
right column. A given normal callout was spoken by SYNCALL only 
once for a given approach. Also, if SYNCALL detected that it 
would deliver a normal callout within +/-50 feet of the altitude 
at which a Decision Height callout, a 100 feet above MDA callout, 
or an MDA callout was to be made or had been made, it supressed 
that normal callout. 

The SYNCALL system was programmed to make deviation callouts 
only if glideslope deviation, localizer deviation, airspeed, or 
sink rate deviated beyond the tolerances that the Pilot-not- 
Flying is required to use for PNF deviation callouts. For exam- 
ple, the Pilot-not-Flying and the F/E were required to call devi- 
ations of +/-5 KTS or more deviation from the selected approach 
airspeed. The SYNCALL system was programmed to make an airspeed 
deviation callout if the deviation reached +/-7 KTS. The SYNCALL 
system was designed with larger tolerances so that it would serve 
as a back-up on deviation callouts for both the PNF and the F/E 
rather than functionally replacing these two crew members for the 
deviation monitoring and callout task. This was done because it 
was feared that the PNF and F/E might not monitor the flight in- 
struments often enough or be left out of the information loop if 
the SYNCALL deviation callouts were not designed to give them 
first chance to detect any deviations. Table 5 lists the devia- 
tion callouts with the flight conditions that triggered them on 
the right and the callout wordings on the left. 

It will be noted that some of the modes for the Ground Prox- 
imity Warning System (GPWS) coincide with conditions that would 
normally trigger deviation callouts by the SYNCALL system, cf. 
"Glideslope” for 1.3 dots or more below the glideslope. In order 
to avoid the aural confusion of concurrent voice messages from 
different sources but stating the same message, the aural warning 
of the GPWS was disabled when the SYNCALL system was in use. The 
SYNCALL system was programmed to perform all of the relevant GPWS 
callouts in addition to the deviation callouts that are not made 
by the GPWS system. In the case of simultaneously occurring 
demands for two or more deviation callouts, the callouts were 
made in sequence. The priority assignment from highest to lowest 
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SYNCALL APPROACH CALLOUTS 



= - TRANSITION LEVEL* 

11,000' MSL (FL 110 IF TRANSITION 
LEVEL IS AT OR BELOW) 



FAF 


G/S 

INTERCEPTION 
(IF MAKING 
ILS APPROACH) 



1,000' AFL 

v / 500' AFL, AIRSPEED ± APPROACH 
VC SPEED AND DESCENT RATE 
400' AFL 

100' 300' AFL 

ABOVE 200' AFL 

MDA 



OUTER MARKER AND 50' RADIO ALTIMETER 

AFL CROSSING ALTITUDE TO TOUCHDOWN AT 

10' INCREMENTS 


DEVIATION CALLOUTS 

AIRSPEED - BELOW 600 AFL, APPROACH SPEED ±7 kt 
RATE OF DESCENT - BELOW 600 AFL 1200 fpm 

BELOW 300 AFL 1000 fpm 
LOCALIZER - ± 1/2 DOT 
GLIDESLOPE - ± 1-1/3 DOT 


Figure 2. - Experimental SYtlCALL (Synthesized Approach Callout) 
System . 
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TABLE 4. SYNCALL CALLOUT TRIGGERING CONDITIONS AND 
CORRESPONDING CALLOUT WORDINGS FOR NORMAL CALLOUTS 


TRIGGERING CONDITION 

Glideslope Interception 
Localizer Interception 
5 s after Outer Marker 
Triggering 

Descending Through 1000 ft AFL 
Descending Through 500 ft AFL 

Descending Through 400 ft AFL 
Descending Through 300 ft AFL * 
Descending Tnrough 200 ft AFL 
Descending Through 100 ft AFL 
Descending Through DH Setting 
on Radio Altimeter and 
Setting < or = 250 ft 
Descending Through 100 feet 
Above Radio Altimeter DH 
Setting & Setting > 250 ft 
Descending Through DH Setting 
on Radio Altimeter and 
Setting > 250 ft 
Descending Through 50 ft Radio 
Altimeter 

" 40 ft Radio Alt. 

" 30 ft " 


CALLOUT WORDING 

GLIDESLOPE INTERCEPT 
LOCALIZER INTERCEPT 

OUTER MARKER ( ) ABOVE FIELD 

ONE THOUSAND FEET ABOVE FIELD 
FIVE HUNDRED FEET. AIRSPEED 

( + / ) , SINK IS ( ) 

FOUR HUNDRED 
THREE HUNDRED 
TWO HUNDRED 
ONE HUNDRED 


DECISION HEIGHT 


ONE HUNDRED FEET ABOVE MDA 


MDA 

FIFTY 

FORTY 

THIRTY 


20 ft " TWENTY 

10 ft " TEN 


KEY: ( ) refers to a specific value spoken by SYNCALL 

e.g. "Five Hundred Feet, Airspeed + 10, Sink is 800," 

DH Decision Height 

MDA Minimum Descent Altitude 


* Note: Theoretically, when the Autoland Mode of the autopilot 
was engaged, indicating a Category II Approach, SYNCALL moni- 
tored the Radio Altimeter for altitude information from 300 
feet to touchdown. Otherwise SYNCALL monitored Barometric 
Altitude from the Captain's Altimeter, which was set to 
Altitude Above Field Level (AFL) until 100 feet. This was in 
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accordance with airline procedure to eliminate erroneous AFL 
altitude readouts due to uneven terrain on final approach. 


was GPWS mode 2a or 2b (Excessive Terrain Closure), glideslope, 
indicated airspeed, sink rate, and localizer. A given deviation 
callout was spoken only once. If, however, the out-of- tolerance 
condition had not been corrected after a specific time lag, the 
deviation callout was repeated. This time lag was 5 s for the 
glideslope, airspeed, and sink rate deviation callouts and 10 s 
for the localizer deviation callouts. For airspeed and sink rate 
deviation callouts, a repetition was triggered after the time lag 
if the deviation was still out of tolerance. For localizer and 
glideslope deviation callouts, a repetition was triggered only if 
after the lag the aircraft was not closing on the desired in- 
tolerance window, i.e. only if the deviation was of the same mag- 
nitude or greater than it had been when the previous deviation 
callout was triggered. This type of repetition logic was 
designed because it takes time for the aircraft to respond to the 
pilot's control input to correct flight path deviations in 
glideslope and localizer. As long as the flightpath was in the 
process of being corrected, the SYNCALL system made no repeti- 
tions of the deviation callout, so as to reduce nuisance cal- 
louts. 

V fiUPfr Message Design 

The wording of the voice messages was selected so as to 
adhere as much as possible to the phraseology used by the crew 
for the approach callouts while applying previous research on 
voice warning message wording (2) to ensure high intelligibility 
even with competing human speech messages. Table 6 lists the 
wordings for each type of SYNCALL approach callout. These word- 
ings were then tested for intelligibility under conditions that 
were more difficult than any expected to occur in a real cockpit. 
(See Appendix A) 
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TABLE 5. SYNCALL CALLOUT TRIGGERINGS AND CORRESPONDING 
CALLOUT WORDINGS FOR DEVIATION CALLOUTS 

TRIGGERING CONDITION CALLOUT WORDING 

After G/S Capture, if 
A/C >/= 1 1/3 dots 

above Glideslope YOU’RE ABOVE THE GLIDESLOPE 

Same as Above, except 
A/C same amount below 

Glideslope (=GPWS Mode 5) YOU’RE BELOW THE GLIDESLOPE 

After Localizer Capture, 
if A/C >/= 1/2 dot right 

of Localizer YOU ARE RIGHT OF THE LOCALIZER 

Same as Above, except 
A/C same amount left 

of Localizer YOU ARE LEFT OF THE LOCALIZER 

A/C Below 600 ft AFL 
and Indicated Airspeed 

> or = 7 kts above 
Approach Speed Setting 

of Autothrottle System AIRSPEED PLUS ( ) 

Same as Above, except 
Indicated Airspeed is 

> or = 7 kts below 

Approach Speed Setting AIRSPEED MINUS ( -) 

If 600-300 AFL & 

Sink > 1200 fpm or 
If < 300 AFL & 

Sink > 1000 fpm SINK IS (- ) 

GPWS Mode 2a or 2b 

Triggered WHOOP WHOOP PULL UP 


KEY: ( ) refers to a value spoken by SYNCALL 

e.g. ’’Sink is 1 300 .” 


Notes: Deviation Callout repetition rate if condition not 

corrected = 5 s for Glideslope, Airspeed, and Sink Rate 
and 10 s for Localizer. Callout Priority from highest to 
lowest was GPWS Mode 2a or 2b (Terrain Closure), Glideslope, 
Airspeed, Sink Rate, Localizer. 
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TABLE 6. EXAMPLES OF SYNCALL CALLOUTS 


LOCALIZER INTERCEPT 
GLIDESLOPE INTERCEPT 
OUTER MARKER 1 4 80 ABOVE FIELD 
1000 FEET ABOVE FIELD 
500 FEET, AIRSPEED +4, SINK IS 1300 
500 FEET, ON SPEED, SINK IS 1200 
400 
300 
200 
100 

100 FEET ABOVE MDA 
MDA 

DECISION HEIGHT 
50 
40 
30 
20 
1 0 

YOU’RE ABOVE THE GLIDESLOPE 
YOU'RE BELOW THE GLIDESLOPE 
YOU ARE LEFT OF THE LOCALIZER 
YOU ARE RIGHT OF THE LOCALIZER 
AIRSPEED MINUS SEVEN 
AIRSPEED PLUS NINER 
SINK IS FIFTEEN HUNDRED 
WHOOP WHOOP PULL UP 


EXPERIMENTAL DESIGN 

In order to study and evaluate the concept of an automatic 
approach callout system using synthesized speech, the experimen- 
tal SYNCALL system was compared to a current airline system of 
approach callouts by the crew - referred to here as PNF callouts. 
For the data to be representative of the range of approach condi- 
tions encountered in the operational environment, several types 
of approaches, varying in crew workload and amount of manual 
flight control, were flown both by captains and first officers. 
For each type of approach, data was collected for each crew for 
both the SYNCALL and PNF approach callouts for both captains and 
first officers as the pilot flying. 
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So as to obtain representative user-pilot performance and 
judgment data, current line pilots, i.e. regular airline pilots 
who routinely fly the aircraft, served as participating pilot- 
subjects. The experimental sessions were incorporated into the 
airline's normal recurrent training that is required periodically 
for air transport pilots. The crew members involved in the study 
and the Federal Aviation Administration were given a guarantee 
that participation in this experimental study would in no way be 
allowed to jeopardize a crew member's training in required 
maneuvers. In all cases, participation in the study was volun- 
tary, and refusal to participate in no way affected a pilot's 
rating or flight status. In fact, the pilots typically expressed 
interest in the study and pleasure at being asked to participate. 
None refused to participate. 

A major advantage of incorporating the experiment into on- 
going recurrent training in an airline simulator was the added 
realism that could be attained from the presence of completely 
operational aircraft systems at all crew member stations and from 
the use of flight procedures flown by pilots who were familiar 
with them and with the aircraft being simulated. It would have 
been difficult to obtain the same degree of realism in a flight 
research simulator since such simulators lack all but the most 
basic flight systems. The major disadvantage of conducting the 
study in the context of airline recurrent training was the set of 
restrictions imposed by the particular types of maneuvers that 
were part of the training syllabus. Since many of these 
maneuvers were different types of approaches, they lent them- 
selves well to the purpose of the study. 

Twenty crews composed of captain, first officer, and flight 
engineer 2 flew approaches of varying difficulty using PNF cal- 
louts for half their approaches and SYNCALL callouts for the oth- 
er half of their approaches. Table 7 lists the different ap- 
proaches in the order that they were flown. Approaches 1, 2, and 
3 comprised "Set 1" and were always flown first. Approaches 4, 
5, and 5 comprised "Set 2" and were always flown last. For a 
given crew, Approach Set 1 was flown with one ,>f the callout sys- 
tems (SYNCALL or PN?) and Approach Set 2 was flown with the oppo- 
site callout system. For Approach Set 1, both captains and first 
officers flew each of the three approaches (1, 2, and 3). For 
Approach Set 2, both captains and first officers flew the first 
two approaches (4 and 5), but only the captains flew the last ap- 
proach (6), the single-engine approach and landing, in accordance 
with the airline procedures. Other than this discrepancy, the 
two approach sets were roughly balanced in that each began with a 
non-precision, manually flown approach. Next came an autopilot 


When scheduling did not permit the participation of a line 
flight engineer, a flight engineer instructor served as flight 
engineer to complete the three-man crew. 
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TABLE 7. APPROACHES FLOWN FOR SYNCALL EVALUATION 


PILOT FLYING APPROACH 
CAPTAIN FIRST OFFICER 

1 LOCALIZER BACK COURSE LOCALIZER BACK COURSE 

SET 

2 DUAL LAND ILS APPROACH ONLY 

1 

3 2 -ENGINE ILS 2-ENGINE ILS 


4 VOR RAW DATA 

SET 

5 SINGLE LAND 

2 

6 1 -ENGINE LANDING 


VOR RAW DATA 
ILS APPROACH ONLY 


assisted, precision approach. Finally there was an approach with 
one or more engines out. A given approach as flown by a captain 
or a first officer was flown to similar but not identical 
minimums due to the fact that captains are qualified to fly to 
lower minimums using more of the autopilot capability than are 
first officers. These small compromises in experimental design 
were deemed well worth the added realism that was gained by fol- 
lowing airline procedures exactly. Table 3 shows the experimen- 
tal design in block form. 

For half the crews, the captain flew Approach Set 1 first, 
followed by the first officer flying Approach Set 1. Then the 
captain flew approaches 4 and 5 of Approach Set 2 followed by the 
first officer flying these two approaches. The last approach to 
be flown was always approach 5, by the captain. For the other 
half of the crews, the first officer flew first for each approach 
set followed by the captain. For half the crews assigned to each 
of these orders, the SYNCALL callouts were used for Approach Set 
1 and the PNF callouts for Approach Set 2. For the other half of 
the crews, the PNF callouts were used for Approach Set 1 and the 
SYNCALL callouts for Approach Set 2. Appendix B contains de- 
tailed descriptions of the four resulting experimental orders for 
the approaches. 3 


3. Captain Walt Estridge and Instructor Pilot Colby Noyes were 
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TABLE 8. EXPERIMENTAL DESIGN 


APPROACH SET 

ORDER SET 1 SET 2 

1 SYNCALL PNF 



CAPT 

F/0 

CAPT 

F/0 

2 

SYNCALL 

PNF 


F/0 

CAPT 

F/0 

CAPT 

3 

PNF 

SYNCALL 


CAPT 

F/0 

CAPT 

F/0 

4 

PNF 

SYNCALL 


F/0 

CAPT 

F/0 

CAPT 


DEVELOPMENT OF PERFORMANCE MEASURES 

The man-vehicle system composed of three flight crew members 
interacting with one another and with the multiple systems of a 
modern jet transport aircraft is extremely complex; the task of 
measuring the effectiveness of different types of approach cal- 
lout systems within that environment shares many of these com- 
plexities* Thus measurements of system effectiveness must take 
these system complexities into account. The measurements must 
also have operational relevance. They must relate directly to 
airline operations in a way that allows the system users (pilots, 
airline management, manufacturers, government regulatory agen- 
cies) to base decisions on system implementation directly on the 
data. Special flight performance measures were devised for this 
study. These measures were computed from the raw data on air- 
craft altitude, airspeed, selected approach speed, localizer de- 
viation in degrees, and glideslope deviation in degrees. This 
raw data was sampled by the simulation computer once every 0.2 s 
and recorded once every 0.8 s during the experimental runs. 

It was assumed that a pilot generally attempts to fly an air- 
craft within certain tolerances or windows and’that as long as 
performance remains within the desired window, no correction will 


instrumental in evolving the formal experimental design into a 
workable flight training syllabus that conformed to the airline 
training requirements. 
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be deemed necessary by the pilot. This assumption is supported 
by the observation that both FA A and airline company regulations 
are written with allowable tolerances. Also, pilots if asked 
will say they fly to tolerances rather than try to keep a needle 
exactly centered or positioned. Indirect support is also sug- 
gested by the need to incorporate an "indifference threshold” 
into control models of human operator performance in order to ob- 
tain an acceptable fit between model-predicted performance and 
observed human operator performance (3)» If this assumption is 
correct, measures of mean deviation from an ideal value for, say, 
airspeed would have little operational relevance to what a pilot 
judges to be good performance. For example, if the pilot at- 
tempts to fly within +/-5 KTS of a selected approach speed, he 
will be satisfied if he remains within + /-2 KTS but will make a 
correction if airspeed deviates by +9 KTS from approach speed. 
It would seem more relevant to keep track of whether a pilot 
flies inside or outside the tolerance window for a given parame- 
ter (airspeed, sink rate, localizer deviation, or glideslope de- 
viation) and, when outside tolerance, to monitor the average and 
maximum deviations from the boundary of the tolerance window - 
not from the ideal value. This reasoning led to the choice of 
four computed operationally relevant measures of flight perfor- 
mance for the approach segment. 

Percent Time out of Tolerance 

For each of the three approach segments (I, II, and III) the 
percentage of the total duration of the segment that was flown 
outside the tolerance window for a given parameter was computed. 
This was done by taking the number of 0.8-s samples for which the 
parameter was outside the tolerance window and dividing by the 
number of 0.8-s samples in the entire segment. 

Maximum Deviation 

The maximum deviation, the deviation with the largest abso- 
lute value, and the direction of that deviation (positive, nega- 
tive, right, left, etc.) for each of the four parameters for each 
of the three approach segments was recorded. This value was then 
corrected to reflect the deviation from the boundary of the 
tolerance window by subtracting the absolute value of the toler- 
ance . 

Mean Eosit i ve Deviation 

Since deviations both right and left of the localizer, above 
and below glideslope, and faster or slower than approach speed 
could occur, mean deviations from the tolerance window boundary 
were computed for each direction. Due to the arbitrary assign- 
ment of signs (+ and -) to the direction of deviation by the 
simulation computer program, the term "positive” deviation is as- 
sociated with real world directionality in an intuitively oppo- 
site manner. So, the arbitrary meanings of "positive” and "nega- 
tive" are given for each of the four parameters below: 
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-2 1 - 



"Positive 

" Deviation 

"Negative" 

Deviation 

Airspeed 

IAS great 

er than V(APPR) 

IAS less 

than V(APPR) 

Sink Rate 

Sink Rate 
tolerance 

greater than 

Not applicable 

Loc. Dev. 

Deviation 

left of course 

Deviation 

right of cour 

G/S Dev. 

Deviation 

below G/S 

Deviation 

above G/S 


The mean positive deviation was computed by taking only those 
0.8-s samples for which the parameter in question was out of 
tolerance in the "positive” direction, summing these deviations, 
and dividing by the number of out-of- tolerance samples. Thus, 
using an operational tolerance window of + /-8 KTS for IAS; if 
measured IAS was 158 KTS and selected V(APPR) was 1 4 1 KTS, then 
deviation from the positive tolerance window boundary of 149 KTS 
(141+8) is +9 KTS. For IAS samples of 153, 156, 150, 148, 149, 
144, and 140 KTS, the mean positive deviation would be (9+7+1 )/3 
or +5.7 KTS. Expressed in terms of deviation from the selected 
approach speed, the mean positive deviation would be +13.7 KTS 
(5.7+8) . 

Mean Negative Deviation 

Mean negative deviation from the tolerance window boundary 
for each flight parameter for each approach segment was computed 
for deviations in the "negative" direction as defined above. 
This was done in the same manner as the computation for "posi- 
tive" deviations. However, this measure was not applicable for 
sink rate since there is no minimum acceptable sink rate for an 
approach. Thus the computed "negative" deviations for sink rate 
were disregarded. 


Flight Performance Parameters 


As stated above, four parameters were measured for deviation 
outside allowable tolerance windows: airspeed, sink rate, local- 
izer displacement, and glideslope displacement. These were 
chosen since they are the parameters which the pilot-no t- f lying 
and the flight engineer are required to monitor during the ap- 
proach; and a comparison of performance on these four parameters 
as a function of callout system would give a measure of compara- 
tive effectiveness of the two callout systems in keeping the air- 
craft within allowable tolerance windows. 

In choosing the operational tolerance window for each parame- 
ter, it was necessary to consider the discrepancies between the 
raw data in the Central Air Data Computer (CADC) (simulated by 
the simulation computer) and the visual indicators used by the 
pilots to determine current airspeed, sink rate, localizer devia- 
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tion, and glideslope deviation. The pilots could not be expected 
to correct excessive airspeed as measured by the CADC, for exam- 
ple, if their airspeed indicators showed them to be within toler- 
ance. Thus an operational tolerance for each flight parameter 
was derived by adding its operating manual ("book") tolerance 
(cf. Figure 3) and its "indicator" tolerance. 

Operational Airspeed Tolerance 

The operating manual for this air carrier calls for IAS to be 
within +/-5 KTS of selected approach speed (V(APPR)) once landing 
flaps have been extended. The permissible error between the di- 
gital autothrottle command setting, which controls the position 
of the orange bug on the airspeed indicator, and the actual posi- 
tion of the orange bug is + /-3 KTS. Thus the airspeed deviation 
as read by the pilot-not-flying and the flight engineer as they 
monitor airspeed from the airspeed indicator could appear as a 5 
KT deviation when in fact it was an 3 XT deviation according to 
data in the CADC. Therefore, so as to count as out of tolerance 
only those airspeed deviations which appeared as deviations to 
the pilot-not-flying and the flight engineer, the book tolerance 
of 5 KTS and the indicator tolerance of 3 KTS were added together 
to produce an operational tolerance of +/-3 KTS. Figure 3 shows 
this pictorially. 

Operational Sink Rate Tolerance 

The book tolerance for sink rate varies with altitude above 
field (AFL). From 1000 feet AFL to 300 feet AFL, sink rate is 
not to exceed 1000 feet per minute (FPM) . Below 300 feet AFL, it 
should not exceed 700 FPM. There is a considerable time lag, ap- 
proximately 2 to 3 s , between instantaneous vertical speed and 
the indication of vertical speed shown on a vertical speed indi- 
cator (V3I). 4 Thus the pilots visual indication of sink rate is 
2 to 3 s old. The recorded sink rate data was taken directly 
from the simulation computer and was thus a derived instantaneous 
vertical speed. So, an indicator tolerance was needed for this 
data that would compensate for the time lag. The computation of 
this tolerance involved the average standard deviation of a ran- 
dom sample of 2.4 s intervals of sink rate. 

Sink rate W3S averaged over 40 sets of three consecutive 


Although Instantaneous Vertical Speed Indicators (IVSI's) 
are installed in the wide-body jet used in this study, a 2 to 3 s 
time lag is designed into the indicators to to smooth the data 
going to the Central Air Data Computers (CADC’s), which in turn 
drive the vertical speed instruments and altimeters in the cock- 
pit. This is done to prevent momentary "opposite direction" 
vertical speed and altitude indications which would otherwise oc- 
cur following changes in aircraft pitch. Pitch changes produce a 
momentary change in local pressure distribution at the static 
ports which results in a driving pressure opposite to that needed 
to give a true indication of direction of vertical speed. 
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OPERATIONAL TOLERANCE: SINK RATE 



1000 TO 300 AFL: 1174 fpm 



1000 

BELOW 


TO 300 AFL 

300 AFL 

BOOK TOLERANCE 

1000 fpm 

700 fpm 

+ INDICATOR TOLERANCE 

174 fpm 

174 fpm 

OPERATIONAL TOLERANCE 

1174 fpm 

874 fpm 


Figure 4. - Operational Tolerance: Sink Rate. 
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0.8-s samples of sink rate, i.e. over 2.4-s intervals. The stan- 
dard deviation for each of these sets of three samples was com- 
puted, Then the mean and standard deviation of these 40 standard 
deviations was computed and these values added together. The 
original 40 sets of three 0.8-s samples of sink rate were taken 
from 10 of the 20 crews, randomly selected. For each of the ten 
crews selected, a 2.4-s interval of sink rate was taken for each 
of the four non-precision approaches (LOC BK CRS and VOR flown by 
Captain and by First Officer) flown by that crew. The altitude 
at which each 2.4 -s interval started was randomly chosen from 
the range 1 - 1000 feet AFL. 

The mean and standard deviation of the 40 resulting standard 
deviations were 57 feet and 117 feet respectively. Their sum of 
174 feet was used as the indicator tolerance for sink rate. Ad- 

ding 174 feet to the book tolerances of 1000 FPM and 700 FPM 
resulted in operational tolerances of: 

1000 - 300 AFL 1174 FPM 

BELOW 300 AFL 874 FPM 

Figure 4 shows this pictorially. 

Operational Localizer Deviation Tolerance 

The book tolerance for localizer deviations is +/-1/3 dot on 
the indicator scale, which is equal to +/-0.42 deg displacement 
from the center of the localizer course. The allowable tolerance 
for the localizer indicator needle is expressed in terms of 
"needlewidths" , The needle referred to is the indicator needle 
which is driven right or left of center depending on whether the 
aircraft is left or right of the localizer course (valid only for 
the "front course" direction) The tolerance is +/-1 needlewidth. 
To convert needlewidth to dots - the unit used by the pilots 
the needlewidth and the distance between center and full deflec- 
tion to one side were measured on an actual Course Direction In- 
dicator (CDI) which displays localizer and glideslope informa- 
tion, Accordingly, 1 needlewidth measured 0,22 dots or 0,28 deg. 
This indicator tolerance was added to the book tolerance of 
+/-1/3 dot (=+/-0,42 deg) to give an operational tolerance of 
+/-0.56 dots (=+/-0.70 deg). In round numbers, this operational 
tolerance is just slightly greater than +/-1/2 dot. See Figure 
5. 

Operational Tolerance for Glideslope Deviations 

The indicator tolerance for glideslope deviation was computed 
in a manner analogous to that for localizer deviation. This 
resulted in the + /-1 needlewidth tolerance being equal to +/-0.18 
dots (=+/-0.07 deg). The book tolerance of +/-1 dot (=+/-0.25 
deg) plus the indicator tolerance gives an operational tolerance 
of +/-1.28 dots (=+/-0.32 deg), or approximately 1 1/4 dots. See 
Figure 6 . 
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OPERATIONAL TOLERANCE: LOCALIZER 


LEFT 

0.7° 

= 0.56 DOT 


RIGHT 

0.7° 

= 0.56 DOT 



BOOK TOLERANCE 1/3 DOT 
+ INDICATOR TOLERANCE 1 NEEDLEWIDTH 
OPERATIONAL TOLERANCE 1/2+ DOT 


Figure 5. - Operational Tolerance: Localizer. 
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OPERATIONAL TOLERANCE: GLIDESLOPE 


CO 

-o 



BOOK TOLERANCE 1 DOT 

+ INDICATOR TOLERANCE 1 NEEDLEWIDTH 

OPERATIONAL TOLERANCE VA DOTS 


Fijure 5. - Operational Tolerance: 


Glideslope . 



MEASURES OF PILOT JUDGMENTS OF SYSTEM PERFORMANCE 

The purpose of administering a formal post-experiment de- 
briefing to the pilots was to obtain in a systematic and objec- 
tive fashion the pilots 1 subjective responses to the two callout 
systems. It was expected that experienced line pilots would be 
able to answer questions about potential advantages and disadvan- 
tages of the two systems that could not be otherwise measured by 
flight performance parameters. By presenting each pilot with an 
identical set of questions and by designing the questions so that 
possible answers were mutually exclusive and yet covered the en- 
tire range of possible answers by the inclusion of an option for 
an "other” response, it was possible to collect data that could 
be submitted to non-parametric statistical tests for signifi- 
cance . 

Deg, l&n o£ EUai, R&- Vx l & . ti riGL M afc s. c;Ul& 

A copy of the de-briefing form exactly as presented to the 
pilots is in Appendix C. Preference grids, a measure that had 
been successfully used with airline pilots in a study of line pi- 
lot preferences for design of cockpit warning systems (4) were 
used to study the effects of type of approach conditions (Night 
VFR, Day VFR, IFR to Weather Minimums, or Abnorm al / Emergency ) on 
whether or not pilots would want a particular callout to be made 
by the Pilot-not-Flying and/or by SYNCALL. A 7-point scale from 
"unsafe” to "highly desirable” with "no preference" in the middle 
was used to compare the present PNF system, the SYNCALL system as 
configured for the study, the PNF system as reconfigured accord- 
ing to the pilot’s own responses on the preference grid concern- 
ing desirable callouts, and the SYNCALL system as reconfigured 
according to the pilot's own preference grid responses. Pilots 
were asked to rate the intelligibility of four types of speech 
they had heard in the simulator on a scale of 0% to lOOjt. The 
four types of speech were 1) SYNCALL synthesized speech, 2) Ter- 
minal Radar Control (TRACON) ATC human voice messages, 3) Tower 
ATC human voice messages, and 4) Pilot-not Flying human voice 
messages (callouts and other crew voice communications). The 
last item was a list of possible modifications to the SYNCALL 
system as configured for the study. Pilots were asked to rate 
each modification on a scale of 1) Highly Desirable, 2) Desir- 
able, 3) No preference, 4) Undesirable, or 5) Unsafe. At the end 
of the list were blank lines with instructions to fill in addi- 
tional modifications the pilots thought they would want. They 
were asked to rate their own suggested modifications using the 
same 5-point scale. This concluded the main debriefing form, 

A second, optional form was used to obtain additional, more 
detailed information about the individual callouts from those pi- 
lots who were interested enough and willing to spend additional 
time. This form consisted of a set of semantic differential 
scales (5) chosen for their relevance to cockpit flight informa- 
tion systems. Several of these scales had been previously used 
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in the study on line pilot preferences for cockpit warning system 
design (4) with consistent results. Others were added specifi- 
cally for this study. Table 9 lists the semantic differential 
scales and Table 10 lists the set of callouts by each system that 
were measured with these scales. For purposes of comparison, the 
current GPWS "Whoop, Whoop. Pull up!" and "Glideslope" voice mes- 
sages were included in the set of callouts to be rated on each of 
the semantic differential scales. These items were included be- 
cause they were then and still are the only examples of electron- 
ically generated speech that have been experienced by all airline 
pilots. Both the de-briefing form and the optional semantic dif- 
ferential form were designed to be self-administered. Finally, 
to preserve annonymity, no names or other personal information 
were requested on either form. 


TABLE 9. SEMANTIC DIFFERENTIAL SCALES 


GOOD. . . . . . 

WORTHLESS. 

LOUD 

ACTIVE. . . . 

NOISY 

SAFE. 

ALERTING. . 
ANNOYING. . 
HELPFUL. . . 
DEPENDABLE 


BAD 

VALUABLE 

SOFT 

pass: IE 

QUIET 

. . . .DANGEROUS 
IMPERCEPTIBLE 

SOOTHING 

. .DETRIMENTAL 
. . .UNRELIABLE 


DISRUPTIVE 


COORDINATING 
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TABLE 10. TYPES OF CALLOUTS MEASURED WITH SEMANTIC DIFFERENTIAL 

AS SPOKEN BY PI LOT-NOT-FL YI NG AND BY SYNCALL 

1 000 -FOOT CALLOUT 

TEN-FOOT CALLOUTS 

HUNDRED FOOT CALLOUTS 

G/S & LOCALIZER INTERCEPT 

FINAL APPROACH FIX & CROSSING ALTITUDE 

AIRSPEED DEVIATIONS 

SINK RATE DEVIATIONS 

G/S & LOCALIZER DEVIATIONS 

DECISION HEIGHT, MDA , & 100 ABOVE MDA 


AS SPOKEN BY GPWS 

"WHOOP WHOOP PULL UP" 
"GLIDESLOPE" 


PROCEDURE 

Each crew received a taped briefing of approximately 
5-minutes length explaining the scope and purpose of the experi- 
ment. If all crew members decided to participate, they were 
given a taped description of the SYNCALL system compared to the 
PNF system accompanied by diagrams on projected transparencies. 
General questions about systems operation were encouraged. Next, 
the pilots heard a taped introductory passage by the speech syn- 
thesizer to f amil'iar ize them with the electronic voice quality 
and "accent" of the speech. This was followed by a recording of 
each type of approach callout in the SYNCALL system with each 
callout pronounced twice. To speed learning of the synthesizer 
accent, the printed text of the speech was displayed concurrent- 
ly, Tables 11 (following) and 6 (in section on Voice Message 
Design) are a reproduction of the printed displays. 

After the 15-minute briefing, the instructor pilot conducted 
the normal pre-simulation briefing on the flight maneuvers to be 
practiced. This was followed by four hours of recurrent simula- 
tor training in accordance with the appropriate set of experimen- 
tal conditions for each crew. The first hour was spent on other 
maneuvers and served as a "warm-up” period before flying the ex- 
perimental approaches, A break of 15 minutes was scheduled 
between Approach Set 1 and Approach Set 2 in accordance with the 
normal training breaks used. To simulate the potential mutual 
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TABLE 11. INTRODUCTORY PASSAGE BY SPEECH SYNTHESIZER PRESENTED 
TO PILOTS FOR F AMILARIZ ATION WITH SYNCALL VOICE 

YOU ARE ABOUT TO HEAR THE SYNCALL SYSTEM SAY 
EACH OF THE APPROACH CALLOUTS SO YOU CAN GET 
FAMILIAR WITH THE SOUND OF THE VOICE AND WITH 
EACH OF THE CALLOUTS. THE CALLOUTS ARE LISTED 
IN ORDER ON THE SLIDE SO YOU CAN LOOK AT THEM 
AS YOU HEAR THEM. EACH CALLOUT WILL BE SAID TWO 
TIMES WITH A SLIGHT PAUSE BETWEEN REPETITIONS. 


masking of concurrent speech messages from Air Traffic Control 
(ATC) and the SYNCALL callouts, a pre-recorded set of Approach 
and Tower transmissions, addressed to other aircraft, were played 
over the cockpit headsets to the pilots. The pilot instructor 
issued clearances in ATC phraseology to the crew, and they read 
back their clearances with the cockpit microphone as they would 
have done in the actual aircraft. The author controlled other 
aircraft ATC message presentation on different simulated ATC fre- 
quencies to assure that the simulated ATC chatter was appropriate 
for the runway in use and the simulator’s position in the air- 
space. The ATC tapes were edited from actual ATC communications 
recorded at the airport that was simulated for the study. 

Flight performance data was collected for each approach by 
the simulation computer. Data collection began at 3000 feet AFL 
and continued until either the initiation of a go-around on a 
missed approach or the touchdown on a landing. The following 
flight parameters were sampled once every 0.8 s: 

1) Indicated Airspeed (IAS) 

2) Selected Approach Speed (from ATS command airspeed) 

3) Altitude Mean Sea Level (M3L) 

4) Glideslope deviation 

5) Localizer deviation 

6 ) Vertical Speed 

7 ) Elapsed Time 

The times of occurrence of each callout together with actual 
values that triggered any deviation callouts were also recorded. 
The cockpit audio including crew conversation, ATC, approach cal- 
louts, and cockpit noise was recorded on audio tape. 

After the simulator session, each pilot completed the de- 
briefing form and discussed the experiment with the author. 
Those pilots who were interested also completed the optional se- 
mantic differential form. Many took it with them and mailed the 
completed form back. Pilots were offered a copy of the final re- 
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port, when published 
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RESULTS 


The results will be presented in sections according to the 
individual measures (flight performance, system reliability, sys- 
tem interference with other systems) that were used to assess 
callout system effectiveness. Because of the large number of 
different measures used, there will be some discussion of each of 
the major types of results in the respective results sections. 
Then the results of the separate measures will be compared for 
consistency. Finally, a general discussion will treat the 
results from different measures as a whole. 

Flight Performance Results 

Analysis of Variance (AOV) was used to test for statistical- 
ly significant effects of 1) type of callout system, 2) pilot 
flying, 3) order of pilot flying, and 4) type of approach. 
Analysis of variance was chosen as a means of testing the data to 
determine the probability that any observed differences in the 
data associated with each of the four effects above could be due 
to chance rather than be caused by those effects themselves. 
Airspeed, sink rate, localizer deviations, and glideslope devia- 
tions were analyzed as appropriate to each approach type for each 
of the four measures: (1) percent time out of tolerance, (2) max- 

imum deviation beyond tolerance, (3) maximum "positive" devia- 
tion, and (4) maximum "negative" deviation. 

First, all five approaches (i-engine approach & landing not 
included here) were compared for the flight parameters they had 
in common, namely airspeed and sink rate, for each of the four 
measures listed above. The final approach was divided into four 
segments for analysis, with the assumption that these segments 
might be flown differently and that the operational tolerances 
for the different segments would be different. The four segments 
were : 

I: Final Approach Fix (FAF) to 1000 AFL 

II: 1000 AFL to 500 AFL 

III: 500 AFL to DH or MDA (depending on type approach) 

IV: DH or MDA to Landing or Go-around 

(depending on how approach was terminated) 


Due to large differences among the five types of approach in 
Segment I, the AOV's were performed only on the data for Segments 
II, III, and IV. While a few callouts were made by SYNCALL or 
the Pilot-not-Flying (PNF) during Segment I (outer marker with 
AFL crossing altitude, localizer deviations, and glideslope devi- 
ations), most of the callouts occur from 1000 AFL on down. Thus 
any differences in performance due to the type of callout system 
would be expected to occur in segments II, III, and IV. 
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Figures 7 and 3 show the different 4-way AOV’s that were per- 
formed. The comparison of all five approaches was done with a 
2x2x2x3 mixed design for approaches 1, 2, and 3 and with a 

2x2x2x2 mixed design for approaches 4 and 5. The reason for the 
division of the analysis into two parts: Approaches 1, 2, and 3 
versus Approaches 4 and 5 was that those pilots who flew one set 
of approaches with one callout system flew the other set of ap- 
proaches with the other callout system. Analysis of all five ap- 
proaches in a 2x2x2x5 mixed design would have resulted in an un- 
balanced design. There were thus 24 four-way AOV’s (3 segments x 
4 measures x 2 parameters) for approaches 1, 2, and 3 and another 
24 four-way AOV’s for approaches 4 and 5. 

Analyses of All Approaches Overall 

Approach Segment II : 1 000 AFL to 500 AFL 

Between 1000 and 500 feet AFL the only factor with signifi- 
cant effects on airspeed and sink rate performance was type of 
approach. Significant differences (p<0.001) were obtained for 
sink rate performance for both sets of approaches (1,2,3) and 
(4,5) for the following measures: 

Percent time out of tolerance 

(Operating tolerance = -1174 FPM ) 

Maximum deviation beyond tolerance 

Mean "positive” deviation beyond operating tolerance 

Significant differences (p<0.001) were obtained for airspeed per- 
formance only for the first set of approaches (1,2,3). The fol- 
lowing measures resulted in significant differences: 

Percent time out of tolerance (Oper Tol = + /-8 KTS ) 

Maximum deviation from Approach Speed 

Mean "positive" deviation from Approach Speed 

The term "mean negative deviation" as applied to the data for 
sink rate and airspeed deserves some comment. During approach 
and landing, only sink rates greater than tolerance are of con- 
cern. "Negative deviation", i.e. rate of climb, has no upper 
limit and was therefore not analyzed. 

For airspeed, there were no instances for any of the 20 crews 
for either pilot for any of the approaches for which airspeed was 
below the minimum operating tolerance of -8 KTS. Since all data 
points were thus equal to 0, analysis of variance was neither ap- 
propriate nor necessary. 

Since the results for mean "positive" deviation were analo- 
gous to those for percent time out of tolerance and for maximum 
deviation, Table 12 presents means (N=40 pilots) just for percent 
time out of tolerance and means for. maximum sink rate deviation 
and Table 13 presents similar data for airspeed for Segment II. 
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PILOT FLYING 


ANALYSIS OF VARIANCE DESIGN FOR 
APPROACH SET 1 
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APPROACH 
£ 1 LOCBKCRS 

S 2 AUTO LAND 
3 2 ENG ILS 

APPROACH 

1 LOC BK CRS 

2 AUTOLAND 

3 2 ENG ILS 

APPROACH 

APPROACH 
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1 LOC BK CRS 

“■ 2 AUTOLAND 

2 AUTOLAND 

3 2 ENG ILS 
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SYNCALL 
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CALLOUT SYSTEM 


‘igure 7. - Analysis of Variance Design for Approach Set I, 
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PILOT FLYING 


ANALYSIS OF VARIANCE DESIGN FOR 
APPROACH SET 2 
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Figure 3. - Analysis of Variance Design for Approach Set II 
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TABLE 12. SINK RATE PERFORMANCE FOR ALL APPROACHES DURING 
SEGMENT II: 1000 FT AFL TO 500 FT AFL 

N = 40 PILOTS 

MEAN PERCENT TIME MEAN MAXIMUM 
TYPE APPROACH OUT OF TOLERANCE DEVIATION 


1 

LOCALIZER BACK COURSE 

39 

t 

1433 

F PM 

2 

DUAL LAND / ILS APPR ONLY 

1 

i 

1131 

FPM 

3 

2-ENGINE ILS 

i 

i 

H?3 

F PM 

4 

VOR RAW DATA 

30 

i 

1405 

FPM 

5 

SINGL LAND / ILS APPR ONLY 

1 

% 

1 174 

FPM 


Note: Tne were no statistically significant differences due to 

Pilot Flying, Order, or Type of Callout System During Segment II 


TABLE 13. AIRSPEED PERFORMANCE FOR ALL APPROACHES 

DURING SEGMENT II: • 1000 FT AFL TO 500 FT AFL 

N = 40 PILOTS 

MEAN PERCENT TIME MEAN MAXIMUM 
TYPE APPROACH OUT OF TOLERANCE DEVIATION 


i 

LOCALIZER BACK COURSE 


34 

a 

fo 

+ 

3 

KT3 

2 

DUAL LAND / ILS APPR 

ONLY 

44 

i 

+ 

3 

KTS 

3 

2-ENGINE ILS 


16 

% 

+ 

5 

KTS 

4 

VOR RAW DATA 


41 

% 

+ 

1 0 

KTS 

5 

SINGL LAND / ILS APPR 

ONLY 

39 

V 

ft 

+ 

12 

KTS 


Note: Tnere were no statistically significant differences for 

Pilot Flying, Order, or Type of Callout System during Segment II 
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It is not surprising that sink rate performance on 
autopilot-assisted ILS and Category II Autoland approaches was 
better than for the manually flown non-precision approaches. It 
is nowever surprising to note that airspeed performance between 
1000 and 500 feet AFL was worse for the Autoland approaches than 
for the manually flown approaches. Possibly the pilot antici- 
pates needs for slight adjustments in pitch and power as the 
landing flaps and gear extend and is in this way aole to track 
desired approach speed better than the autothrottle system which 
can only react after airspeed deviates from programmed toler- 
ances. No factors other than type of approach were significant 
for Segment II. 

Approach Segment III : 500 AFL t_o Qjr. MPA 

Between 500 feet AFL and DH or MDA (depending on type of ap- 
proach) there were significant effects for type of approach 
( p< 0 , 03 1 ) , callout system (p<0.05), and pilot flying (p<0.05). 
Table 14 (a-d) shows means (N = 40 pilots) for the sink rate and 
airspeed data for Segment III. The only significant effect of 
callout system was for maximum sink rate deviation for approaches 
4 and 5 with SYNCALL approaches resulting in no instances of sink 
rate exceeding the maximum operating tolerance of -1174 fpm and 
the PNF callout system resulting in a mean maximum deviation of 
27 FPM beyond the operating tolerance of -1174 fpm. No signifi- 
cant differences in sink rate due to callout system occurred for 
Approaches 1, 2, and 3. Pilot flying had a significant effect 
(p<0.05) on percent time out of tolerance ( + /-3 KTS ) for airspeed 
for approaches 1, 2, and 3. Captains on the average were out of 
tolerance on airspeed 22 % of the time while First Officers were 
out of tolerance only 10% of the time. As with data for the 
previous segment, the type of approach flown had a significant 
effect (p<0.0(£T) on both airspeed and sink rate. In this seg- 
ment, between 500 AFL and DH or MDA, performance was better for 
the autopilot-assisted ILS approaches than for the manually flown 
non-precision approaches. 
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TABLE 14a. AIRSPEED PERFORMANCE FOR SET 1 APPROACHES DURING 
SEGMENT III: 500 FT AFL TO DH OR MDA 








N = 40 

PILOTS 

FACTOR 



MEAN 

PERCENT 

TIME 

MEAN 

MAXIMUM 




OUT 

OF TOLERANCE 

DEVIATION 

CALLOUT 


SYNCALL 


13 

% . 


+ 

3 

KTS 

SYSTEM 


PNF 


20 

% 


+ 

4 

KTS 

PILOT WHO 


CAPTAIN 


15 

% 


+ 

3 

KTS 

FLEW FIRST 


FIRST OFFICER 


18 

% 


+ 

4 

KTS 

PILOT 


CAPTAIN 


22 

i 


+ 

4 

KTS 

FLYING 


FIRST OFFICER 


10 

% 


+ 

3 

KTS 

APPROACH 

1 

LOCALIZER BACK 

COURSE 

34 

% 


+ 

6 

KTS 

TYPE 

2 

DUAL LAND/ILS 

APPR ONLY 

3 

% 


4 * 

1 

KT 


3 

2-ENGINE ILS 


13 

% 


4 - 

3 

KTS 



TABLE 1 4b . AIRSPEED PERFORMANCE FOR SET 2 APPROACHES DURING 
SEGMENT III: 500 FEET AFL TO DH OR MDA 

N = 40 PILOTS 

FACTOR ' MEAN PERCENT TIME MEAN MAXIMUM 

OUT OF TOLERANCE DEVIATION 


CALLOUT 

SYNCALL 

20 

% 

4 - 

3 

KTS 

SYSTEM 

PNF 

12 

% 

4 - 

3 

KTS 

PILOT WHO 

CAPTAIN 

19 

% 

4 - 

3 

KTS 

FLEW FIRST 

FIRST OFFICER 

14 

% 

4 - 

3 

KTS 

PILOT 

CAPTAIN 

13 

i 

+ 

2 

KTS 

FLYING 

FIRST OFFICER 

19 

% 

4 - 

5 

KTS 

APPROACH 4 

VOR RAW DATA 

31 

% 

+ 

5 

KTS 

TYPE 5 

SINGL LAND/ILS 

APPR ONLY 2 

t 

4 - 

2 

KTS 
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TABLE 14c 


SINK RATE PERFORMANCE FOR SET 1 APPROACHES DURING 
SEGMENT III: 500 FEET AFL TO DH OR MDA 


N = 40 PILOTS 

FACTOR MEAN PERCENT TIME MEAN MAXIMUM 

OUT OF TOLERANCE DEVIATION 


CALLOUT 

SYNCALL 

6 

% 

56 

FPM 

SYSTEM 

PNF 

6 

% 

51 

FPM 

PILOT WHO 

CAPTAIN 

5 

% 

44 

FPM 

FLEW FIRST 

FIRST OFFICER 

7 

% 

63 

FPM 

PILOT 

CAPTAIN 

7 

% 

70 

FPM 

FLYING 

FIRST OFFICER 

5 

% 

37 

FPM 

APPROACH 1 

LOCALIZER BACK COURSE 

1 1 

% 

106 

FPM 

TYPE 2 

DUAL LAND/ILS APPR ONLY 

0 

% 

0 

FPM 

3 

2-ENGINE ILS 

5 

% 

53 

FPM 



TABLE 1 4d . SINK RATE PERFORMANCE FOR SET 2 APPROACHES DURING 
SEGMENT III: 500 FEET AFL TO DH OR MDA 







N = 40 

PILOTS 

FACTOR 


MEAN PERCENT 

TIME 

MEAN MAXIMUM 



OUT OF 

TOLERANCE 

DEVIATION 

CALLOUT 

SYNCALL 


0 

% 


0 

FPM 

SYSTEM 

PNF 


6 

% 


27 

FPM 

PILOT WHO 

CAPTAIN 


6 

% 


14 

FPM 

FLEW FIRST 

FIRST OFFICER 


i 

% 


13 

FPM 

PILOT 

CAPTAIN 


3 

% 


16 

FPM 

FLYING 

FIRST OFFICER 


4 

% 


1 1 

FPM 

APPROACH 4 

VOR RAW DATA 


6 

% 


23 

FPM 

TYPE 5 

SINGL LAND/ILS 

APPR ONLY 

1 

% 


3 

FPM 


Note: For sink rate, Maximum Deviation refers to the deviation 
BEYOND the operation tolerance of 1174 fpm. 
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Approach Segment JJL: J2H SUL MM ±0. LanliHg. SUL Q^-Ax^JOA 

Between DH or MDA and Landing or Go-Around , there were also 
significant effects for type of approach (p<0.05) and callout 
system (p<0.05). Table 15 (a-d) shows the means (N=40 pilots) 
for the sink rate and airspeed performance data for Segment IV, 
In this segment, the only significant difference resulting from 
type of callout system was for percent time out of tolerance for 
airspeed, and the difference was in the opposite direction from 
that which was expected - performance was worse with SYNCALL (35% 
time out of tolerance) than with PNF callouts (21% time out of 
tolerance) . This reversal occurred only for the second set of 
approaches (Numbers 4 and 5). One possible explanation is that 
the crews who were using PNF callouts for these approaches had 
already experienced SYNCALL for the first set of approaches and 
may have experienced some learning transfer to fly tighter toler- 
ances with SYNCALL which carried over into their performance 
without SYNCALL in the second set of approaches. Further exami- 
nation of the raw data would be needed to determine whether such 
transfer effects may have actually occurred. It should be noted 
that airspeed performance was generally worse for the second set 
of approaches than for the first, possibly due to fatigue. All 
experimental runs were flown between 20:00 and 24:00 Central 
Time, causing the second set of approaches to be flown between 
22:30 and 24:00, 
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TABLE 15a. 

AIRSPEED PERFORMANCE FOR 

SET 1 APPROACHES DURING 


SEGMENT IV: DH OR MDA TO 

LANDING OR 

GO 

-AROUND 



N = 40 

PILOTS 

FACTOR 

MEAN PERCENT TIME 


MEAN MAXIMUM 


OUT OF 

TOLERANCE 


DEVIATION 

CALLOUT 

SYNC ALL 

13 £ 


+ 4 KTS 

SYSTEM 

PNF 

21 £ 


+ 4 KTS 

PILOT WHO 

CAPTAIN 

16 £ 


+ 4 KTS 

FLEW FIRST 

FIRST OFFICER 

i 1 £ 


+ 3 KTS 

APPROACH 1 

LOC ALIZEWR BACK COURSE 

23 £ 


+ 4 KTS 

TYPE 2 

DUAL LAND/ILS APPR ONLY 

4 £ 


+ 2 KTS 

3 

2-ENGINE ILS 

25 £ 


+ 6 KTS 


TABLE 15b. 

AIRSPEED PERFORMANCE FOR 

SET 2 APPROACHES DURING 


SEGMENT IV: DH OR MDA TO 

LANDING OR 

GO 

-AROUND 



N = 40 

PILOTS 

FACTOR 

MEAN PERCENT TIME 


MEAN MAXIMUM 


OUT OF 

TOLERANCE 


DEVIATION 

CALLOUT 

SYNCALL 

35 £ 


+ 14 KTS 

SYSTEM 

PNF 

21 £ 


+ 1 1 KTS 

PILOT WHO 

CAPTAIN 

25 £ 


+ 11 KTS 

FLEW FIRST 

FIRST OFFICER 

25 £ 


+ 13 KTS 

APPROACH 4 

VOR RAW DATA 

23 £ 


+ 9 KTS 

TYPE 5 

SINGL LAND/ILS APPR ONLY 

35 £ 


+ 16 KTS 


Summary o f 4 - wav AO V T s 

While there were statistically” significant differences for 
type of callout system, the absolute magnitude of the differences 
was generally small. This is especially true for the sink rate 
data. Also, the differences due to type of approach were of far 
greater magnitude than those due to either the type of callout 
system or the pilot flying. Order of pilot flying had no signi- 
ficant effects on sink rate or airspeed performance. 
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TABLE 15c. SINK RATE PERFORMANCE FOR SET 1 APPROACHES DURING 
SEGMENT IV: DH OR MDA TO LANDING OR GO-AROUND 




N 

= 40 PILOTS 



FACTOR 

MEAN 

PERCENT TIME 

MEAN 

MAXIMUM 


OUT 

OF TOLERANCE 

DEVIATION 

CALLOUT 

SYNCALL 

8 

% 

50 

FPM 

SYSTEM 

PNF 

3 

% 

59 

FPM 

PILOT WHO 

CAPTAIN 

3 

% 

65 

FPM 

FLEW FIRST 

FIRST OFFICER 

8 

% 

79 

FPM 

APPROACH 1 

LOCALIZER BACK COURSE 

’3 

% 

113 

FPM 

TYPE 2 

DUAL LAND/ILS APPR ONLY 

2 

% 

16 

FPM 

3 

2-ENGINE ILS 

10 

% 

53 

FPM 


TABLE 1 5d , 

SINK RATE PERFORMANCE FOR 

SET 2 

APPROACHES DURING 


SEGMENT IV: DH 

OR MDA TO 

LANDING OR 

GO-AROUND 





N 

= 40 

PILOTS 


FACTOR 


MEAN PERCENT 

TIME 

MEAN 

MAXIMUM 



OUT OF 

TOLERANCE 

DEVIATION 

CALLOUT 

SYNCALL 


5 

% 


129 

FPM 

SYSTEM 

PNF 


9 

% 


65 

FPM 

PILOT WHO 

CAPTAIN 


5 

% 


1 12 

FPM 

FLEW FIRST 

FIRST OFFICER 


9 

% 


H7 

FPM 

APPROACH 4 

VOR RAW DATA 


9 

% 


166 

FPM 

TYPE 5 

SINGL LAND/ILS 

APPR ONLY 

2 

% 


2 

FPM 


Analyses of Indi 
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Figure 9 shows the different AOV's that were performed on the in- 
dividual approaches. The design for analysis of individual ap- 
proaches was a 2x2x2 across subjects AOV. 

All together this resulted in 3 segments x 4 measures x 4 
parameters for each of the 3 precision approaches plus 3 segments 
x 4 measures x 3 parameters for the Localizer Back Course ap- 
proach plus 3 segments x 4 measures x 2 parameters for the VOR 
approach for a total of 204 3-way AOV 1 s . 

When individual approaches were analyzed, Type of Callout 
System (SYNCALL or PNF) was the only factor that had statistical- 
ly significant effects on flight performance. Furthermore, sig- 
nificant differences in flight performance resulted only for the 
non-precision approaches (Localizer Back Course and VOR) only in 
the lower altitude segments (III: 500 AFL to MD A ; and IV: MDA to 
Land ing/Go- Around ) and only for airspeed and sink rate perfor- 
mance - not course tracking performance (Localizer and Glideslope 
where applicable). In each case, flight performance was better 
with SYNCALL than with PNF callouts. 

Airspeed Deviations During Localizer Back Course Approach 

Table 16 shows airspeed performance for Segment IV (MDA to 
Land ing/G o- Around ) for the Localizer Back Course Approach for 
each callout system. There were 19 such approaches flown by ei- 
ther a Captain or a First Officer for which SYNCALL was used and 
17 approaches for which PNF callouts were used. 5 Percent time 
out of tolerance for the SYNCALL approaches was 13$ compared to 
34$ time out of tolerance for the PNF approaches. This differ- 
ence was significant (F=4.30, df=1,28, p<0.05) Similarly, fewer 
approaches were flown out of tolerance at all (5) with SYNCALL 
compared to PNF (12). 

To discover in more detail what was occurring with airspeed 
performance during these approaches, a tally was taken to obtain 
the number of approaches that were in tolerance and outside 
tolerance at an earlier point in the approach (500 feet) when the 
normal 500 foot callout with airspeed and sink rate is made and 
again at the beginning of the segment for which performance 
differences occurred (MDA). Tables 17 a and b show number of ap- 
proaches in and out of tolerance for SYNCALL and PNF callouts at 
500 AFL and the corresponding data taken at MDA, A Chi-Square 
analysis resulted in no significant correlation between callout 
system and airspeed performance at 500 AFL (X(2)=0.07, df=1, 
p>0.10). But the Cni-Square for the data at MDA was significant 
( X (2 ) = 5 • 56 , df = 1 , p< 0 , 02 ) , Thus, while there was no difference 


5. If the crew on a given approach initiated a go-around be- 
fore reaching MDA (as sometimes happened) then there was no data 
collected for that approach for that crew. This is the reason 
for the unequal N number of approaches for the two callout sys- 
tems. 
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PILOT FLYING 


DESIGN FOR ANALYSIS OF VARIANCE FOR 
INDIVIDUAL APPROACHES 



Figure 9. - Design For Analysis of Variance for Individual Ap- 
proaches . 
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TABLE 16. AIRSPEED PERFORMANCE FOR LOCALIZER BACK COURSE 

APPROACH DURING SEGMENT IV: MDA TO L ANDING/GO-A ROUND 


CALLOUT SYSTEM 

MEAN PERCENT TIME 
OUT OF TOLERANCE 

NUMBER 
OUT OF 

OF APPROACHES 
TOLERANCE 

SYNCALL 

(19 APPROACHES) 

13 

% 


5 

PIL0T-N0T-FLYING 
(17 APPROACHES) 

34 

% 


12 


in airspeed performance at 500 AFL that was associated with type 
of callout system, by the time MDA was reached, airspeed perfor- 
mance with SYNCALL was better than with PNF callouts only. Fig- 
ure 10 graphs the mean airspeed deviation from Selected Approach 
Speed at 500 AFL and at MDA for SYNCALL and PNF approaches. 
Average airspeed deviation for PNF approaches tends to increase 
slightly (+6.5 KTS +/-4.9 KTS at 500 AFL increasing to +7.9 KTS 
+/-4.3 KTS by MDA), In contrast, average airspeed deviation for 
SYNCALL approaches tends to decrease (+6.3 KTS +/-3.9 KTS at 500 
AFL decreasing to +4.6 KTS +/-3.2 KTS by MDA) Analysis of Vari- 
ance yielded no significant effect on average airspeed deviation 
due to either altitude (500 vs MDA) or callout system (SYNCALL vs 
PNF), but the probability of an interaction between the two ap- 
proached significance ( Fraax =1,85, df=17, p<0.10) Tnis analysis 
taken together with the significant effect on airspeed perfor- 
mance associated with type callout system in the lower segment 
from MDA to Landing or Go-Around suggests that airspeed perfor- 
mance improved during the descent when SYNCALL was in use but did 
not improve during the same part of the descent when only the PNF 
callouts were in use. 

^ I n. K Rate Deviations During the VQR Approach 

Table 18 shows the sink rate data for Segment III (500 AFL to 
MDA) for the V0R Raw Data Approach. Analysis of Variance for ef- 
fects of callout system, pilot flying, and order of pilot flying 
on sink rate performance for this approach yielded a significant 
difference (F=5.01; df=1,31; p<0.05) due to callout system on the 
maximum deviation beyond operational tolerance. The effect of 
callout system on percent time out of tolerance approached signi- 
ficance ( F = 4 ,06; df= 1 , 3 1 ; p = 0.053 ) When SYNCALL was in use, there 
were no instances for the 19 approaches flown with SYNCALL of 
sink rate exceeding the operational tolerance of 1174 FPM. Of 
the 19 approaches flown with PNF callouts 4 had sink rate devia- 
tions in excess of tolerance. The mean maximum deviation of 
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MEAN AIRSPEED DEVIATION, kt 


MEAN AIRSPEED DEVIATIONS FOR LOCALIZER 

BACK COURSE APPROACH 



ALTITUDE 


Figure 10. - Mean Airspeed Deviations For Localizer Back 
Approach . 



TABLE 17a. NUMBER OF APPROACHES INSIDE AND OUTSIDE TOLERANCE 
FOR AIRSPEED DURING LOCALIZER BACK COURSE APPROACH 
AT 500 FEET AFL 


SYNC ALL PILOT-NOT-FLYING 


INSIDE TOLERANCE 12 


i 0 


OUTSIDE TOLERANCE 7 


7 


Note: Data were available for only 36 of the 40 approaches flown 
since 3 approaches terminated in a go-around before the aircraft 
reached MDA and since data were not available for a 4th approach. 


TABLE 17b. NUMBER OF APPROACHES INSIDE AND OUTSIDE TOLERANCE 
FOR AIRSPEED DURING LOCALIZER BACK COURSE APPROACH 
AT MDA (= 239 FEET AFL) 



SYNCALL 

PILOT-NOT-FLYING 

INSIDE TOLERANCE 

16 

3 

OUTSIDE TOLERANCE 

3 

9 


these four "out-of-tolerance" approaches was 1 4 1 2 FPM +/-167 FPM 
or 238 FPM beyond the operating tolerance of 1174 FPM, Percent 
time out of tolerance for all approaches together including those 
that were always within tolerance was 0 % for SYNCALL approaches 
compared to 9 % for PNF approaches. Finally, for those 4 PNF ap- 
proaches that were out of tolerance the mean percent time out of 
tolerance was 37. 3 % • 

l- Engins An&r _o_ap h and Landing Airspeed Per formance 

As stated above, the 1-Engine Approach and Landing is flown 
by Captains only and thus had to be analyzed separately. It also 
differs from other types approaches in the procedures for 
airspeeds to be flown at various segments of the approach. The 
other approaches are flown from 1000 feet to landing using Ap- 
proach Speed, abbreviated V(APPR). 6 On aircraft equipped with 

5. The pilots compute this mentally by adding one half the 
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TABLE 18. SINK RATE PERFORMANCE FOR VOR RAW DATA APPROACH 

DURING SEGMENT III: 500 FEET AFL TO MD A (435 FEET) 


CALLOUT SYSTEM 
SYNCALL 

(19 APPROACHES) 

PILOT-NOT-FLYING 
(19 APPROACHES) 


NUMBER OF APPROACHES 
OUT OF TOLERANCE 

0 

4 


MEAN MAXIMUM 
DEVIATION 


1412 FPM + /- 167 
(FOR 4 APPROACHES) 


an Auto Throttle System (ATS), like the one simulated in this 
study, the pilots set an orange airspeed command bug on their 
airspeed indicators to their chosen V(APPR). Tney then use this 
bug to visually assist them if flying manually; also, the ATS, if 
engaged, will automatically maintain this commanded airspeed. A 
callout of airspeed referenced to Approach Speed is made at 500 
AFL as a cross check. For the 1-Engine Approach, Approach Speed 
is defined differently. It is the 0 deg flaps/slats extended 
minimum maneuvering speed for the landing gross weight of the 
aircraft, abbreviated V(0 deg/EXT). This higher approach speed 
is maintained until a visual landing commit point of 700 feet AFL 
in case the crew should have to make a go-around and thus need 
the added speed for a climb-out with only one engine. If they 
decide a landing can be assured from 700 AFL then the Captain be- 
gins to reduce airspeed so as to arrive at 100 AFL with a speed 
of V(Ref) plus the conventional wind additives of one half runway 
wind speed plus gust velocity. The orange command airspeed bug 
is left at the higher Approach Speed of V(0 deg/EXT) for the 
duration of the approach. Thus, in contrast to other types of 
approaches, the pilot now attemps to fly an airspeed well below 
that indicated by the orange bug, 10 to 20 KTS in the aircraft 
simulated for this study. 

The pilot-not- flying in this instance knows that the captain 
will be reducing airspeed below the approach speed shown by the 
orange bug and makes allowance for this when calling out routine 
airspeed checks or when making an airspeed deviation callout. 
The SYNCALL system, on the other hand, obtains its Approach Speed 
setting from the orange ATC command bug setting. Thus, when the 


headwind plus the added speed factor of gusts to V reference with 
the restriction that Approach Speed will never be less than 
V(Ref) plus 5 KTS or greater than V(Ref) plus 20 KTS, 
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captain slows the aircraft to the slower approach speed, the SYN- 
CALL system would be inappropriately triggered to make an 
airspeed deviation callout. 

In order to determine whether such inappropriate deviation 
callouts had an adverse effect on airspeed performance during the 
Single Engine Approach and Landing, airspeed was measured refer- 
enced to the desired approach speed of V(Ref) + wind when the 
aircraft was at 100 feet AFL, the point at which procedures call 
for the speed to be at this slower approach speed. Airspeed per- 
formance would be adversely affected if, for the SYNCALL ap- 
proaches, airspeed was excessively above V(Ref) + wind as com- 
pared to the PNF approaches. Table 19 shows mean Indicated 
Airspeed (IAS) minus V(Ref) + wind additives at 100 AFL for the 
10 approaches with SYNC ALL and the 9 approaches with PNF cal- 
louts. While SYNCALL approaches averaged 9.4 KTS above the 
desired approach speed and PNF approaches averaged only 6.5 KTS 
above approach speed, this difference, when tested for statisti- 
cal significance by T-Test was not significant (T=1.01, df=17, 
p>0.10). Also, the number of approaches that were flown outside 
tolerance were almost equal with SYNCALL and with PNF callouts: 4 
out of 10 approaches out of tolerance with SYNCALL compared to 5 
out of 9 approaches out of tolerance with PNF callouts. However, 
the standard deviation of the airspeed performance with PNF cal- 
louts was only +/-3.6 KTS compared to +/-7.8 KTS for the airspeed 
performance with SYNCALL. This difference in variability of 
airspeed performance was significant by two-tailed test for 
differences between independent variances (F=4.69; df=9,8; 
p<0,05) Tnus the captains' airspeed at 100 feet during the 
Single-Engine Approach and Landing was more variable with SYNCALL 
than with PNF callouts. This is certainly an adverse effect on 
airspeed performance from an operational standpoint. 


TABLE 19. AIRSPEED PERFORMANCE FOR 1-ENGINE APPROACH 
AND LANDING - CAPTAIN ONLY 
AIRSPEED MEASURED AT 100 FEET AFL 



MEAN AIRSPEED 

STANDARD 


DEVIATION 

DEVIATION 

CALLOUT SYSTEM 

/ 



SYNCALL 

(10 APPROACHES) 

+ 9.4 KTS 

+/- 

7.8 KTS 

PIL0T-N0T-FLYING 
(9 APPROACHES) 

+ 6.5 KTS 

+/- 

3.6 KTS 
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Summary of Individual Approach Results 

In summary, there were specific instances of improved perfor- 
mance on airpseed and sink rate when SYNCALL was in use for the 
non-precision approaches. However, when SYNCALL made inappropri- 
ate airspeed deviation callouts for the 1-Engine Approach, 
airspeed performance became more variable and there was a trend 
for the pilot-flying to erroneously heed the inappropriate cal- 
lout. 
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SYNCALL And Pilot- Not-Flying Callout Performance 


Tile cockpit audio recordings for ten of the twenty crews were 
analyzed to obtain several types of data. The ten crews were 
chosen such that half the crews had flown with SYNCALL first and 
therefore with Approach Set 1, and the other half had flown with 
SYNC ALL last and therefore with Approach Set 2, £ach callout 
made either by the SYNCALL system or by a crew member was tran- 
scribed. Wnen both SYNCALL and PNF callouts were made for a 
given item, the relative order of the two callouts was noted. 
For the approacnss flown using PNF callouts, the audio of tne 
SYNCALL system had been burned off in tne cockpit. However, the 
SYNCALL system continued to function, albeit silently as far as 
the crew was concerned. These callouts made by SYNCALL were 
recorded on audio tape mixed with the ongoing cockpit audio for 
the PNF approaches. Thus it was possible to determine, for the 
PNF approaches, whether SYNCALL would nave made a given callout, 
had it been operational and to compare the reliability of SYNCALL 
to PNF callouts in terms of O whether a given callout was made 
or not and 2 ) which system made the callout first: SYNCALL or 
PNF. 7 Finally, a tabulation was made of other concurrent audi-o 
events that occurred in the cockpit whs \ SYNCALL callouts would 
have been made for the approaches that used PNF callouts. 

Reliabi lity o f Callouts 

Table 20 shows the reliability of PNF and of SYNCALL for mak- 
ing normal approach callouts in the simulator. For the 53 ap- 
proaches for which PNF callouts were used, a total of 746 cal- 
louts were required. 507 of these or 724 were made by the 
Pilot-not-Flying. For the 53 approaches with SYNCALL, a total of 
635 normal callouts were required. 8 609 of these or 374 were 
made by SYNCALL. It should be noted that there were no ap- 
proaches for which no callouts at all were made, for either sys- 
tem. Rather, for some approaches not all the callouts were made, 
and this was true for both systems. The difference in number of 
callouts made by SYNCALL and by PNF was significant ( X ( 2 ) = 9 1 , 2 4 , 
df=1 , p < 0 , 0 0 2 ) . 

Since any conclusions regarding tne relative reliability of a 

7. This ingenious idea was the suggestion of Gene Tomlinson, 
simulation computer programmer for the project. 

8. Tne difference of 746 required callouts the 53 PNF ap- 
proaches compared to the 635 required callouts for the 53 SYNCALL 
approaches resulted from the differences in go-around altitudes 
used by different pilots for those approaches that were terminat- 
ed in go-arounds, For example, a go-around at 450 feet would re- 
quire altitude callouts only to 500 feet while a go-around at “*50 
feet would require altitude callouts all the way down to 200 
feet. 
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TABLE) 20, RELIABILITY OF SYNCALL AND OF PILOT-HOT-FLY ING FOR 
MAKING NORMAL APPROACH CALLOUTS IN THE SIMULATOR 

NAS CALLOUT MADE ? TOTAL PERCENT 


CALLOUT SYSTEM 

YES 

HO 



SYNCALL 

(53 APPROACHES) 

509 

7 6 

635 

3 7 * 

PILOT-MOT-FLYING 
(53 APPROACHES) 

507 

239 

745 

7 2 i 


system like SYNCALL compared to Pilo t-not -Fly ing callouts were to 
be made on the basis of data collected in a simulation of actual 
flight, the reliability of PNF callouts in the simulator was com- 
pared to the reliability of PNF callouts in tne actual aircraft 
on the. line. This comparison could only be performed for normal 
callouts and could not be done for matched sets of approaches 
since most approaches flown on the line are flown in visual 
weatner conditions even if oy instrument procedures. By con- 
trast, all but the single-engine approach in the simulator are 
flown with simulated instrument weather conditions down to deci- 
sion height or MD A . If anything, however, this should tend to 
make PNF callout reliability on the line lower than in the simu- 
lator, if low callout reliability is associated with better 
weather conditions on the approach. Table 21 shows the reliabil- 
ity of normal PNF callouts being made on the line and in the 
simulator with 7 9£ of the normal callouts made on the line com- 
pared to 72 i of the normal callouts made in tne simulator, Tne 
difference in tne numoer of callouts made in each case was signi- 
ficant (X( 2 )= 9 . 91 , df=1, p<0,002) with the normal callouts on the 
line actually oeing more reliably made than the normal callouts 
made in the simulator during the experiment. Thus, if anything, 
PNF normal callout s would oe expected to be closer in reliability 
to SYNCALL normal callouts when made on the line during actual 
flight operations. 

In an attempt to maxe the line data and the simulator data 
more comparable, the percentage of normal callouts made under the 
three conditions: i) PNF on t. he line, 2) PNF in the simulator 
during the experiment, and 3) SYNCALL in the simulator during the 
experiment were re-calculated, deleting from consideration all 
the MD A and DH callouts, since these were rarely required on the 
line due to tne good weather (4 of the 54 line approaches were 
flown in weather close to the published minimums). Table 22 
shows the resulting percentages with the previous percentages 
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TABLE 21. RELIABILITY OF PILOT - WOT - FLYING ON THE LINE AND IN 
THE SIMULATOR FOR MAKING NORMAL APPROACH CALLOUTS 
FROM 1000 FEET AFL TO 10 FEET RADIO ALTIMETER 



WAS CALLOUT 

res 

MADE ? 
NO 

TOTAL 

PERCENT 

ON THE LINE 
(54 APPROACHES ) 

5 9 5 

139 

335 

79 i 

Ii'I THE SIMULATOR 
(53 APPROACHES 

432 

172 

60 4 

72 i 


that did include MD A and DH callouts, for comparison. No sta- 
tistical analysis of this three-way comparative data was per- 
formed since the validity of such a comparison is doubtful. 
Rather, it is simply interesting to note that, the 3 3 £ reliability 
of PNF normal callouts on the line would appear to be very close 
to the 35/S reliability of SYNCALL normal callouts in the simula- 
tor. And this in turn brings into question the importance of the 
statistically significant difference between PNF and SYNCALL nor- 
mal callout reliability in the simulator. This point will be 
discussed further in the Discussion section. 


TABLE 22. 

RELIABILITY OF 
PILOT-NOT-FLYIN 
AND 3 Y SYNCALL 

NORMAL APPROACH CALLOUTS 
G ON THE LINE AND IN THE 
IN THE SIMULATOR 

MADE BY 
SIMULATOR 



MDA 4 DH CALLOUTS 
INCLUDED 

MDA 

EXCLUDING 
& DH CALLOUTS 

ON THE LINE 


79 % 


3 3 t 

PNF IN THE 

SIMULATOR 

72 % 


74 i 

SYNCALL IN 

THE SIMULATOR 

37 % 


35 % 


The reliability of the deviation callouts made by SYNCALL and by 
PNF is shown in Table 23. While SYNCALL was designed to repeat a 
deviation callout if the problem was not corrected, for purposes 
of analysis, such repetitions were scored as belonging to the 
same instance of a required callout. Tnus, if for a given devia- 
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tion, the callout system made one or more callouts, that required 
callout was scored as having been made once. It was noted that 
SYNCALL repeated deviation callouts for a given deviation far 
more frequently than did the PNF and this was the main reason for 
the chosen method of counting number of callouts made. 


TABLE 23. RELIABILITY OF DEVIATION CALLOUTS BY SYNCALL AND BY 
PILOT-NOT-FLYING IN TH£ SIMULATOR 


CALLOUT SYSTEM 

WAS CALLOUT 
7ES 

HADE ? 
NO 

TOTAL 

PERCENT 

SYNCALL 

75 

3 

33 

90 % 

PILOT-NOT-FLYING 

39 

50 

39 

49 % 


(SYNCALL OFF) 


Data are totals from 53 approaches for each callout system. 


When PNF deviation callouts were used, 49 % of the instances 
requiring deviation callouts had deviation callouts made by the 
pilot-not-flying. This is compared to 90£ of the instances re- 
quiring deviation callouts for which callouts were made by the 
SYNCALL system when it was in use. Tne difference in numbers of 
callouts, made by the two systems was significant ( X ( 2 ) =4 1 . 63 , 
d f = 1 , p<0 , 002 ) , 

Table 24 shows the number of deviation callouts made by the 
pilot-not-f lying when SYNCALL was not in use compared to the 
number of deviation callouts made by the pilot-no t- fly ing when 
SYNCALL was operational. The PNF made only 24£ of tne required 
deviation callouts when SYNCALL was also making deviation cal- 
louts, compared to 49J of the required deviation callouts when he 
had sole responsibility for making these callouts, Tnis differ- 
ence in numbers of callouts made in the two conditions was also 
significant (X(2)=7.4l, df=1, p<0.01) Thus, while SYNCALL was far 
more reliable than PNF for making deviation callouts, SYNCALL 
also seems to have decreased the reliability of the PNF callouts 
themselves even though the pilots were instructed to make devia- 
tion callouts if required when SYNCALL was in use and even though 
SYNCALL was designed with larger tolerances than those used by 
the pilot-not-flying. 

Table 25 lists the individual normal callouts with the 
numbers of each that were or were not made by PNF in the simula- 
tor, by SYNCALL in the simulator, and for comparison, by PNF on 
the line. The asterisks indicate those callouts for which there 
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TABLE 24. RELIABILITY OF DEVIATION CALLOUTS BY PILOT-NOT-FLYING 
WITH SYNCALL AND WITH NO SYNCALL 



WAS CALLOUT 
YES 

MADE ? 
NO 

TOTAL 

PERCENT 

PILOT-NOT-FLYING 
WITH SYNCALL 

20 

63 

83 

24 

% 

PILOT-NOT-FLYING 
NO SYNCALL 

39 

50 

39 

49 

% 


Data are totals from 53 approaches for each condition. 


was a significant difference (p < 0.01) between the number of 
callouts made by SYNCALL and by PNF in the simulator, Tne cal- 
louts for which SYNCALL was more reliable than PNF appear to fit 
into certain categories: 

Critical Descent Altitudes 
Dec ision Height 
MDA 

100 above MDA 

Outer Marker crossing altitude 

Flight Path position 
Localizer Intercept 
Glideslope Intercept 

Routine read-out of performance values 
IAS at 500 feet 
Sink Rate at 500 feet 

Ten- foot altitude callouts 
50 
40 
30 

10 * The difference for the 20 foot callout 

could not be tested since two cells of 
the Chi-Square had expected values less 
than 5 . 

Finally, it is interesting to note that both in the simulator 
and on the line, the PNF makes a 100 foot callout with fair reli- 
ability, 73? in the simulator and 38 5 on the line compared to 775 
by SYNCALL, This is particularly interesting since the flight 
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TABLE 25. RELIABILITY OF INDIVIDUAL NORMAL APPROACH CALLOUTS 
MADE BY 3YNCALL AND BY PI LOT-N OT-FLY I NO IN THE 
SIMULATOR AND ON THE LINE 





IN 

SIMULATOR 




ON THE LINE 



PNF 



SYN 

CALL 


PNF 




X £3 

NO 

j» 

sig? 

YES 

NO 

t 

YES 

NO 

l 

CALLOUT 











LOC INT 

1 4 

29 

33 

* 

39 

0 

100 

_ 

- 

~ 

G/S INT 

25 

7 

7 9 

* 

30 

i 

9 7 

- 

- 

- 

0 M 


23 

1 0 

70 

* 

32 

i 

97 

- 

- 

- 

0 A ALT 

12 

21 

35 

* 

32 


97 

- 

- 

- 

10 0 0 

FEET 

50 

2 

95 


5 1 

1 

93 

55 

9 

36 

5 00 F 

EET 

45 

5 

33 


44 

3 

35 

57 

6 

90 

IAS 3 

500 

27 

20 

57 

* 

44 

3 

35 

33 

26 

59 

SNK 3 

500 

30 

19 

5 1 

* 

44 

3 

35 

2 7 

37 

42 

400 


3 0 

22 

53 


32 

19 

5 3 

49 

1 4 

73 

300 


34 

1 2 

74 


36 

17 

34 

55 

3 

33 

200 


34 

13 

72 


23 

1 5 

59 

55 

9 

35 

100 


24 

9 

73 


20 

5 

77 

5 6 

3 

33 

1 00+MDA 

15 

7 

6 9 

# 

2 1 

0 

1 00 

0 

1 

0 

MD A 


10 


43 

it 

21 

0 

1 00 

0 

i 

0 

or 












DH 


15 

1 4 

52 

* 

25 

0 

100 

3 

6 0 

5 

50 


2 5 

9 

74 


25 

0 

1 00 

59 

5 

92 

40 


24 

9 

73 

it 

25 

0 

1 00 

5 5 

3 

33 

30 


23 

3 

74 

it 

24 

0 

1 00 

50 

4 

94 

20 


2 5 

5 

33 


21 

0 

100 

52 

2 

97 

10 


23 

r 

0 

79 

i 

19 

0 

100 

53 

i 

93 

manual for 

tne 

participating airline 

does 

not call for a 

100 foot 

callout. 

None- 

the- 

less, 

the author 

in d e 

signing 

the 

SIN 

CALL sys- 

tem and th 

e line pilots 

both in 

the 

airplane and 

in 

the 

simulator 


appear to have generalized tne concept of 100-foot callouts to 
include a callout at 100 feet. 

Relative Tim ing o f N ormal Callouts jfnaji_e bj_ PN £ .and, by 3 Y.NC ALfU 

Table 2b shows the relative timing of normal callouts made by 
PNF compared to those that would have been made by SYNC ALL for 
the PNF callout approaches. Of all 745 callouts that were re- 
quired, 5 i were/would not have been made by SYNCALL, compared to 
25J tnat were not made by PNF. 8£ of the required callouts were 
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-53- 


not mads by either system. When both the PNF callout was made 
and tne SYN CALL callout was/would have been made, a total of 622 
of the callouts, the distribution was closely balanced between 
the PNF callout coming first ( 2 H%) and the SYNCALL callout coming 
first (29%) with a few instances of the two callouts being start- 
ed simultaneously (9$). 


TABLE 26. RELATIVE TIMING OF NORMAL APPROACH CALLOUTS MADE BY 
PILOT-NOT-FLYING COMAPARED TO SAME CALLOUTS MADE BY 
SYNCALL FOR 53 APPROACHES. THE CREW DID NOT HEAR THE 
SYNCALL CALLOUTS. NORMAL CALLOUT PROCEDURES WERE USED 


MISSING SYNCALL CALLOUT 

NUMBER 

41 

PERCENT 
5 ; 

PNF CALLOUT 

BEFORE SYNCALL 

132 

24 

•f 9 

PNF CALLOUT 

SAME TIMS AS SYNCALL 

55 

9 

b 

PNF CALLOUT 

AFTER SYNCALL 

216 

29 

% 

MISSING PNF 

CALLOUT 

134 

25 

n> 

MO CALLOUT BY EITHER SYNC ALL/PNF 

57 

3 

jo 

TOTAL NUMBER 

OF CALLOUTS 

74 5 

1 00 

■ •f 

JO 


Other audio information that overlapped SYNCALL callouts 

Table 27 shows the percentages of SYNCALL callouts that oc- 
curred at the same time as various other types of audio informa- 
tion, Again, tne data was taken from the approaches for which 
PNF callouts were in use and SYNCALL callouts were recorded on 
the audio tape only, not heard in the cockpit. 50% of the SYN- 
CALL callouts did occur/would have occurred at the same time as 
some other audio information. Tne interference with ATC communi- 
cations between the controller and the simulated flight was 
minimal (32 of the crew's transmissions to ATC and 32 of ATC 1 s 
transmissions to the crew. The overlap with other ATC chatter 
was higher (132) but probably acceptable since pilots only occa- 
sionally report learning useful information from listening to 
background ATC chatter (such as routing to expect and, occasion- 
ally critical traffic information). However, 233 of the callouts 
would nave occurred during crew checklists and other callouts. 
This relatively large percentage could have masked eitner the 
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SYNCALL callout or the crew voice message, had it actually oc- 
curred. 

In contrast, the Pilot-not-Flying when making Approach Cal- 
louts was observed to sequence these with other crew checklist 
calls so as to minimize overlap. Like SYNCALL, however, the PNF 
callouts often overlapped with Other ATC. 


TABLE 27. PERCENT OF SYNCALL CALLOUTS WHICH WERE OVERLAPPED 

WITH OTHER, SIMULTANEOUS COMPETING AUDIO INFORMATION 


CREW TO ATC 3 % 
ATC TO CREW 3 % 
OTHER ATC 18 % 
CREW CHECKLISTS/CALLS 23 % 
OTHER SOUNDS 3 % 
NO OVERLAP 50 % 


Timin g Accuracy of Ten - Foot Callouts by SYNCALL 

Figure 11 shows the minimum, average, and maximum errors in 
the timing of ten-foot callouts made by SYNCALL in terms of the 
difference between the called altitude and the actual aircraft 
altitude when the callout started. The altitude errors are shown 
as a function of sink rate. Thus, the higher the rate of des- 
cent, the less timely is the information given by a ten-foot cal- 
lout. For example, with a sink rate of 200 FPM, the ten-foot 
callout for 30 feet would, on the average, start when the air- 
craft had actually descended to 27.5 feet. For a sink rate of 
1000 FPM, the same callout would start, on the average, when the 
aircraft had descended to 21,5 feet. Thus for high rates of des- 
cent ten-foot callout altitude information by SYNCALL was not 
timely. Figures 12, 13, and 14 contain the actual data points 
from which the three curves were derived. Tne timliness of the 
corresponding PNF callouts can be inferred by examining the rela- 
tive timing of each PNF ten-foot callout with that of the 
corresponding SYNCALL ten- foot callout and tabulating these rela- 
tive time categories (Before, Same Time As, After) for different 
levels of sink rate. When this analysis was done, no systematic 
relationship was found, suggesting factors other than sink rate 
for the timeliness of PNF 10- foot callouts. 
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In appropriate Callouts by S Y N C A L L 

Certain callouts ware made by the SYNCALL system at inap- 
propriate times or contained inappropriate information. Most of 
these were artifacts related to the instructional aspects of fly- 
ing approaches in a flight simulator. For example, pilots heard 
an immediate "Localizer Intercept", each time the simulator posi- 
tion was reset to a position on the localizer track. Another 
anomaly was the message "100 feet above MDA" if the instructor 
pilot used the altitude reset capability. These problems would 
not be expected in an actual aircraft. 

Othe, SYNCALL callouts were made occasionally for situations 
which had been unintended in the design of the system. Unlike 
the inappropriate callouts listed above, these had some relevance 
to the current flight situation, and, in fact, some of the pilots 
upon hearing them remarked that they gave useful information. 
SYNCALL called out the Outer Marker and crossing altitude any 
time the aircraft passed over the outer marker while being vec- 
tored by ATC. "Localizer Intercept" was called out any time the 
aircraft was vectored through the localizer at an altitude less 
than 3003 AFL. Localizer deviation callouts were then made fol- 
lowing such unintended localizer intercept callouts. While the 
pilots found the unintended localizer intercept callouts useful, 
the ensuing deviation callouts were generally perceived as a nui- 
sance , 

Finally, there was one instance of SYNCALL callouts that were 
not only inappropriate but which, while giving true information, 
presented this information in a way which implied exactly the op- 
posite of the intended message to the crew. This was the case of 
airspeed deviation callouts which called airspeed as being exces- 
sively slow when in fact the pilot-flying was purposely slowing 
the aircraft. for the single-engine approach, discussed above in 
the section on flight performance results. 

Callouts Mad e bv PNF _B_u_t Not S YNC ALL 

During the experiment, it was observed that the Pilot-not- 
Flying made other callouts appropriate and useful to the approach 
that the SYNCALL system had not been designed to make. Some of 
these had been purposely omitted from SYNCALL system design to 
avoid nuisance callouts. One such was deviation callouts right 
or left of VOR course, Tne simulator Course Deviation Indicator 
(CDI) did not distinguish Localizer and VOR signals; thus no 
difference in sensitivity for course deviation callouts could be 
programmed for VOR course deviations as compared to Localizer 
course deviations. The PNF did however, take this difference 
into account when making deviation callouts during the VOR ap- 
proach. SYNCALL was purposely designed to make airspeed and sink 
rate deviation callouts only between 500 and 100 feet AFL to 
avoid nuisance callouts. The PNF, however, occasionally judged 
it necessary to make such callouts above or below this altitude 
range. The result of the inappropriate negative airspeed devia- 
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ALTITUDE ERROR OF TEN-FOOT CALLOUTS 
BY SYNCALL AS A FUNCTION OF SINK RATE 

(AT LEAST 5 SAMPLES PER DATA POINT) 



Function of Sink Rate. 



MINIMUM ALTITUDE ERROR, ft 


MINIMUM ALTITUDE ERROR BY SINK RATE 
FOR TEN-FOOT CALLOUTS BY SYNCALL 



0 200 400 600 800 1000 1200 1400 

SINK RATE, fpm 


Figure 12. - Mini mini Altitude Error by Sink Rate for Ten-Foot 


Callouts by SYNCALL . 



MEAN ALTITUDE ERROR, ft 


MEAN ALTITUDE ERROR BY SINK RATE FOR 
TEN-FOOT CALLOUTS BY SYNCALL 
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Figure 13. - Mean Altitude Error by 3ink Rate For Ten-Foot 
louts by 3YMC/ILL . 
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MAXIMUM ALTITUDE ERROR, ft 


MAXIMUM ALTITUDE ERROR BY SINK RATE 
FOR TEN-FOOT CALLOUTS BY SYNCALL 
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Figure 14. - Maximuin Altitude Error by Sink Rate for Ten-Foot 


Callouts by SYNCALL. 



tion callouts during the single-engine approach was often posi- 
tive airspeed deviation callouts by the PNF, reflecting the actu- 
al situation if it appeared that the pilot-flying was erroneously 
heeding the SYNCALL messages and not slowing the aircraft. Fi- 
nally, the PNF called out "Below MDA" or "Above MDA" if either 
occurred after reaching MDA but before the runway was in sight. 
Since SYNCALL had no information about visual acquisition of the 
runway, it would have been unable to make such callouts appropri- 
ately. 

Summary of Callout Performance Data 

SYNCALL was more reliable than PNF for making deviation cal- 
louts and also for making normal callouts. However, comparison 
with line data for normal callouts suggests that the PNF normal 
approach callouts on the line would be close in reliability to 
those made by SYNCALL. SYNCALL systematically missed certain 
callouts that were made by PNF, and neither system made all the 
callouts. Crew communications with ATC had minimal overlap with 
SYNCALL, however, a relatively large percentage of SYNCALL cal- 
louts overlapped crew checklist and callout messages. 

Pilot Judgment Data 

Table 28 shows the percent of the 40 pilots who found each 
category of callout by SYNCALL (normal and deviation) "useful and 
desirable" for each of the four types of approach conditions. 
Nearly all (85%) of the pilots who flew SYNCALL during this 
evaluation wanted the system to make normal altitude callouts 
during approaches when the weather was down to IFR minimums. 73% 
wanted SYNCALL to make deviation callouts under these same condi- 
tions. For the other conditions, the percent of pilots who want- 
ed SYNCALL callouts of the two types ranged from 49% for devia- 
tion callouts during Day VFR conditions to 78% for normal alti- 
tude callouts during abnormal or emergency conditions. For N=40 
pilots, Chi-sqaure analysis with 1 degree of freedom yields 
p<0.01 for any condition in which 71% or more pilots wanted SYN- 
CALL. Thus the data support a strong pilot preference for SYN- 
CALL during approaches with IFR weather, or with abnormal or em- 
ergency conditions. For night VFR conditions, only normal alti- 
tude callouts received this strong preference. There is no such 
preference for SYNCALL during day VFR operations. 
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TABLE 28. PERCENT OF PILOTS WHO FLEW SYNCALL THAT WANTED IT TO 
MAKE NORMAL CALLOUTS AND OR DEVIATION CALLOUTS 
FOR FOUR TYPES OF APPROACH CONDITIONS. N = 40 PILOTS 
(20 CAPTAINS AND 20 FIRST OFFICERS) 


APPROACH CONDITION 



NIGHT 

VFR 

ABNORMAL/ 

EMERGENCY 

DAY VFR 

IFR TO MINIMUMS 

NORMAL 
ALTITUDE 
CALLOUTS 
(10 TYPES) 

71 

% 

78 % 

57 % 

85 % 

DEVIATION 
CALLOUTS 
(4 TYPES) 

56 

% 

69 % 

49 % 

73 % 

MEAN FOR ALL 
CALLOUTS 

67 

% 

75 % 

55 % 

82 % 
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TABLE 29. PERCENT OF PILOTS WHO FLEW SYNCALL THAT WANTED THE 
PILOT-NOT-FLYING TO MAKE NORMAL CALLOUTS AND OR 
DEVIATION CALLOUTS FOR FOUR TYPES OF APPROACH 
CONDITIONS 

N = 40 PILOTS (20 CAPTAINS AND 20 FIRST OFFICERS 


APPROACH CONDITION 


NORMAL 

ALTITUDE 

NIGHT 

VFR 

ABNORMAL/ 

EMERGENCY 

DAY VFR 

IFR TO MINIMUMS 

CALLOUTS 
(10 TYPES) 

57 

% 

68 % 

51 % 

68 % 

DEVIATION 

CALLOUTS 
(4 TYPES) 

52 

% 

67 % 

47 % 

65 % 

MEAN FOR ALL 

CALLOUTS 

54 

% 

68 % 

50 % 

67 % 


Table 29 shows the corresponding data for Pilot-not-Flying 
callouts. Using the Chi-Square probabilities from above, there 
were no conditions that reached the 0.01 level of significance. 
Thus, the pilots as a group were neither in favor of nor against 
pilot-not-f lying callouts. 

Table 30 shows pilots' perceived intelligibility ratings for 
the synthesized voice compared to the human voice messages of the 
ATC tapes and of the pilot-not-f lying callouts. Interestingly, 
after this initial exposure, the intelligibility ratings for the 
synthesized voice were close to those for actual ATC communica- 
tions. Many pilots noted independently that the electronic voice 
quality of the synthesizer made it very distinctive and left no 
doubt regarding the source of the voice. 

Figure 15 shows the mean ratings for all callouts on four of 
the semantic differential scales for each of the callout systems 
(SYNCALL and PNF ) . These data come from the 28 of the 40 pilots 
or 70% who volunteered to complete this optional form on their 
own time. Differences between ratings for SYNCALL and PNF cal- 
louts were significant for all four scales (Informative- 
Distracting t=-3.23, P < 0 . 0 5 ; Helpful-Detrimental t=-5.12, 
p<0.002; Dependable-Unreliable t=7.98, P<0.002? Disruptive- 
Coordinating t=-6.42, p<0.002; df=8 for each scale). 


67 



SEMANTIC DIFFERENTIAL RATINGS OF 
CALLOUTS BY 14 CAPTAINS AND 14 
FIRST OFFICERS 
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Figure 15. - Senantic Differential Ratings of Callouts by 14 


Captains and 14 First Officers. 



SEMANTIC DIFFERENTIAL RATINGS OF THE 
GPWS "GLIDESLOPE" AND THE GPWS 
“WHOOP WHOOP! PULL UP!" BY 14 
CAPTAINS AND 14 FIRST OFFICERS 
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Figure 15. - 3e nan tic Differential Ratings of the GPWS 
"Glideslope" and the GPWS "Whoop Whoop! Pull Up!" by 14 Captains 
and 14 First Officers. 
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TABLE 30. PERCEIVED INTELLIGIBILITY (PILOT JUDGMENTS) OF 

DIFFERENT TYPES OF SPEECH HEARD DURING SYNCALL STUDY 

SYNCALL (SYNTHESIZED VOICE) 75 % 

ATC COMMUNICATIONS (TAPED, LIVE) 76 % 

PILOT-NOT-FLYING CALLOUTS 88 % 


While both systems were judged as informative, helpful, 
dependable, and coordinating, callouts given by PNF were judged 
as more informative, more helpful, and more coordinating than the 
same callouts by SYNCALL. On the other hand, callouts by SYNCALL 
were judged as more dependable than the same callouts by PNF. 

For purposes of comparison to an existing system that pro- 
duces voice messages on the approach to landing, the pilots rat- 
ings of the GPWS "Whoop, Whoopi Pull Upl" and "Glideslope" mes- 
sages are given in Figure 16. 

The next pilot judgment data was obtained from a follow-up 
debriefing form that was mailed to the 28 pilots who participated 
in the optional semantic differential form. Of these, 13 pilots, 
or 46%, responded in a period from 3 to 9 months after the exper- 
iment. The follow-up debriefing form is reproduced in Appendix 
IV. 

Table 31 shows the modal responses for tolerance values for 
each of the deviation callouts. These pilots' recommendations 
for sink rate, localizer, and glideslope tolerances matched those 
of the SYNCALL system as designed for the study. However, their 
recommendations for airspeed tolerances were different from the 
SYNCALL tolerance of +/- 7 KTS . They suggested +10 KTS and -5 
KTS , 

Table 32 shows these pilots' recommendations for active zones 
for SYNCALL deviation callouts. Their recommendations matched 
the original design of the SYNCALL system. 

Finally, Table 33 shows these pilots' mean and median ratings 
of the recommended modifications suggested by one or more of the 
40 pilots who participated in the study. In the column on the 
far right is listed, for each recommendation the number of the 13 
pilots who judged that recommendation to be "essential in order 
for you to accept SYNCALL in your cockpit". 
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TABLE 31. LINE PILOT PREFERENCES FOR SYNCALL DEVIATION CALLOUT 
TOLERANCES . ALL DATA ARE MODES. 

RECOMMENDED TOLERANCE 

AIRSPEED V (APPR) +10 OR - 5 

SINK RATE 

2000 TO 1000 AFL 2500 FPM 

1000 TO 300 AFL 1200 FPM 

BELOW 300 AFL 1000 FMP 

LOCALIZER 1/2 DOT 

GLIDESLOPE 1 1/3 DOTS 


Note: One captain suggested 1 1/2 dots for localizer back course 
and 2 dots for a VOR approach. Another captain suggested 2 dots 
glideslope to 500 feet AFL and then 1 dot when below 500 AFL. 


TABLE 32. LINE PILOT PREFERENCES FOR SYNCALL DEVIATION CALLOUT 
REPETITION RATES AND ACTIVE ZONES. ALL DATA ARE MODES 


REPETITION RATE ACTIVE ZONE 
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TABLE 33. PILOT RATINGS OF THEIR OWN SUGGESTED MODIFICATIONS 
TO SYNCALL USING THE SCALE: 


unsafe: 

UNDESIRABLE 

NO PREFERENCE 
OR CAN , T DECIDE 

DESIRABLE 

HIGHLY 

DE3IRABL 

5 

• 4 

— 3 

2 



SUGGESTIONS WERE MADE BY ALL 40 PILOTS. RATINGS WERE OBTAINED 
FROM 13 PILOTS WHO RESPONDED TO SECOND DEBRIEFING BY MAIL UP TO 
9 MONTHS AFTER HAVING FLOWN SYNC ALL IN THE EXPERIMENT. 

SUGGESTION MEAN SD MEDIAN NUMBER 

JUDGED 
ESSENTIAL 
(N = 13 ) 

OBTAIN F A A & COMPANY AGREEMENT 
NOT TO USE COCKPIT VOICE RE- 
CORDINGS OF SYNCALL FOR 


DISCIPLINARY ACTIONS 

1.17 

0.53 

1 

10 

ON/OFF SWITCH 

1.23 

0.52 

1 

10 

ELIMINATE NUISANCE CALLOUTS 

1 . 40 

0.70 

1 

5 

PROVIDE TEST SWITCH 

1.54 

0.65 

1 

6 

MANUAL VOLUME CONTROL 

1.69 

0.95 

1 

5 

CALLOUT FOR "BELOW MDA" 

2.00 

1.15 

2 

3 

INCREASE TOLERANCE ON 
AIRSPEED DEVIATION CALLOUTS 

2.03 

1.04 

2 

4 


INHIBIT AIRSPEED AND SINK 
RATE DEVIATION CALLOUTS 


DURING ABNORMAL OR 
APPROACHES 

EMERGENCY 

2.09 

1 .51 

1 

4 

MINIMIZE FREQUENCY 
DEVIATION CALLOUTS 

OF 

2.15 

0 . 30 

2 

2 

ELIMINATE GPWS AND 
IT WITH SYNCALL 

REPLACE 

2.15 

0 . 90 

2 

3 

REPEAT IO-FOOT CALLOUTS IF 
AIRCRAFT BALLOONS DURING FLARE 

2.15 

0 . 90 

2 

3 
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WARN IF AIRSPEED IS 


BELOW ALPHA SPEED 

2.17 

1.11 

2 

1 

CLARIFY SYNC ALL AND PILQT- 
NOT FLYING RESPONSIBILITIES 

2.23 

1 .09 

3 

2 

CALL LOG CAPTURE AND THEN 
LOCALIZER INTERCEPT 

2.23 

0.33 

2 

1 

SHORTEN MESSAGE 

2.25 

0.75 

2 

2 

SPEED UP SYNCALL SPEECH RATE 

2.37 

1.13 

2 

4 

ADD OTHER WARNINGS, E.G. ENGINE 
FIRE, LOSS OF CABIN PRESSURE 

2.46 

1 . 20 

2 

1 

AUTOMATICALLY CHANGE SYNCALL 
PARAMETERS ACORDING TO 
TYPE OF APPROACH 

2.46 

1 .05 

2 

2 

FOR NON-PRECISION APPROACHES 
MAKE SINK RATE DEVIATION 
CALLOUTS HAVE TOP PRIORITY 

2.46 

1 .05 

3 

2 

MORE ALERTING VOICE QUALITY 
FOR DEVIATIONS THAN FOR NORMAL 
ALTITUDE CALLOUTS 

2.52 

0.37 

3 

2 

INCREASE TOLERANCE FOR 
GL IDE3L0PE DEVIATIONS 

2.35 

0.90 

3 

2 

HAVE SYNCALL SAY "CHECK FLAPS' 1 
AT 1000 FEET AFL 

3.00 

1 .00 

3 


HAVE SYNCALL READ OUT FLAPS 
SETTING AT 1000 FEET AFL 

3.00 

1.29 

4 

2 

INCREASE TOLERANCE FOR 
LOCALIZER DEVIATIONS 

3.03 

0.79 

3 

1 

ADD IMMEDIATE ACTION 
CHECKLIST ITEMS 

3.03 

1.32 

4 

1 

HAVE SYNCALL MAKE DEVIATION 
CALLOUTS ONLY 

3.09 

■i.i4 

4 

0 

HAVE SYNCALL READ OUT 
FLIGHT GUIDANCE SYSTEM MODES 
AFTER LAND MODE IS ARMED 

3 . 1 5 

0.99 

3 

1 
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DELAY CALLOUT IF CREW IS 
MAKING RADIO TRANSMISSION 
UNTIL TRANSMISSION COMPLETED 

3.23 

1 .09 

3 

4 

INCREASE VOLUME' 

3.35 

0.50 

3 

1 

HAVE SYNCALL MAKE NORMAL 

ALTITUDE CALLOUTS ONLY 

3.46 

0.97 

4 

1 

} 

WITH THE 50-F00T CALLOUT, 

CALL AIRSPEED AND SINK RATE 
AND ELIMINATE 40,30,20,10-FOOT 
CALLOUTS 

3.54 

1 .27 

4 

1 


Several of t tie pilots noted independently that the airspeed 
deviation callouts in particular were repeated too often and be- 
fore they could slow the airspeed to within tolerance. They 
found this to be a nuisance and distraction. This judgment was 
supported in the rating data on the list of modifications that 
the pilots suggested for SYNCALL on the de-briefing forms. Of 
tne 31 suggested modifications, "Eliminate Nuisance Callouts" 
ranked 3rd with a mean of 1.4 on a scale of 1 to 5 for highly 
desirable to unsafe and had a median response of 1 (highly desir- 
able). Tne suggestion "Minimize Frequency of Deviation Callouts" 
ranked 9th out of 31 with a mean of 2, 1 5 and a median response of 
2 (desirable). It will be remembered that more pilots in select- 
ing those callouts by SYNCALL that they though useful and desir- 
able, chose normal callouts than did deviation callouts. Tne 
perceived nuisance of excessive repetition rates for deviation 
callouts could possibly explain the smaller percentage of pilots 
who wanted deviation callouts by 3YNCALL compared to those who 
wanted normal callouts by SYNCALL. There was, incidentally, no 
correlation between individual pilots’ flight performance in each 
of the four parameters and the desirability of deviation callouts 
in each of the parameters. If anything, there was a trend for 
those pilots with greater percent time out of tolerance in a 
given parameter to be more likely to find deviation callouts for 
that parameter useful and desirable. The item "Increase Toler- 
ance on Airspeed Deviation Callouts" ranked 7th with a mean of 
2.0B and a median response of 2 (desirable). Also, the airspeed 
tolerance was the only tolerance that the pilots who completed 
the follow-up debriefing form wanted changed. And the change 
they recommended was to increase the tolerance on the positive 
side from +7 to +10 KTS while decreasing the tolerance on the 
negative side from -7 to -5 KTS. 

Summary of Pilot Judgment- D ata 

Clearly the pilots who evaluated SYNCALL generally found the 



system desirable for some but not all types of approach condi- 
tions. They seem to indicate that they would find it a useful 
aid but would not want to have to use it for every approach. An 
on/off switch was judged "highly desirable" and "essential" by 10 
of the 13 pilots who responded to the suggested modifications. 
It should be noted here that there were no differences between 
captains and first officers in the way they evaluated SYNCALL; 
nor did the particular set of approaches the pilots flew with 
SYNCALL have any statistically significant effect on their 
evaluation of the system. This suggests that the pilot judgment 
data is reliable and that there is considerable agreement among 
pilots regardless of the position they fly or the specific condi- 
tions of the approaches they flew using SYNCALL. 
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DISCUSSION 


Implica tions of Improved Flight Performance 

The results of this study suggest that the concept of syn- 
thesized voice approach callouts should be developed further. 
Flight performance with SYNC ALL making the callouts was better 
during non-precision approaches than when only the current PNF 
callouts were used. The parameters of flight performance that 
were better when associated with SYNCALL were airspeed and sink 
rate. Airspeed (as an input to estimated ground speed) and sink 
rate constitute the raw data used by a pilot to fly a desired 
glide-path during a non-precision approach which, by definition, 
has no direct glide-path information displayed to the pilot. A 
non-precision approach is considered more difficult to fly just 
because of the lack of direct glide-path information as compared 
to the easier precision approach which provides glide-path infor- 
mation displayed directly to the pilot. Thus the use of SYNCALL 
was associated with improved flight performance for a type of ap- 
proach that is more difficult to fly. 

By comparison data for the precision approaches, showed no 
statistically significant differences in flight performance as a 
function of callout system. If SYNCALL were a benefit only for 
non-precision approaches, what might be the relative gain to air- 
line operations as a whole by the implementation of a properly 
designed synthesized voice approach callout system? 

Non- pr ec is ion approaches result in a large percentage of the 
air carrier approach and landing accidents and incidents, 53% ac- 
cording to the NTSB study (1). If a system which improves flight 
performance for this type of approach in particular made a suffi- 
ciently large contribution to lowering the over-all approach and 
landing accident rate it might pay for its installation. Of 
course, one would have to consider other solutions as well, in- 
cluding elimination of the non-precision approach from airline 
operations, different types of cockpit displays, and different 
types of flight information displayed to the pilots. A compara- 
tive cost analysis of such alternatives is beyond the scope of 
this study. 

There is no single reasonable conclusion that can be reached 
from the experimental observation that SYNCALL was associated 
with better flight performance only for non-precision approaches 
- approaches which demand high Levels of attention by the PF and 
by the PNF who is monitoring. It could be that the higher atten- 
tion workload that is usually but not exclusively associated with 
non-precision approaches is the underlying factor whose effect 
could be mitigated by SYNCALL. If this is correct, then SYNCALL 
would be expected to be helpful during any approach with high at- 
tention workload. 
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from desired flight parameters but not to make routine callouts. 
Altitude callout reliability by PNF was comparable to that of 
SYNCALL. Also 23% of the SYNCALL callouts overlapped crew check- 
list callouts. ' In contrast, the PNF minimized this overlap by 
sequencing altitude callouts between checklist callouts whenever 
possible. Thus one cannot attribute improved flight performance 
to any greater reliability of SYNCALL over PNF for routine alti- 
tude callouts. And it is possible that SYNCALL altitude callouts 
actually interferred with pilots' comprehension of checklist cal- 
louts. Pilots did, in fact, rate SYNCALL less "coordinating 11 
than PNF. All this argues against automatic altitude callouts by 
SYNCALL. 


The case for deviation callouts by SYNCALL is very different. 
They were substantially more reliable when made by SYNCALL than 
by PNF. Also, because they occurred infrequently compared to al- 
titude callouts, their interference with crew checklist callouts 
would be minimal and thus could be an acceptable price in in- 
creased audio workload to pay to the benefit of timely and reli- 
able alerts to flight performance deviations. 

Occasionally, however, an altitude callout can function as a 
warning of an unnoticed deviation. One can speculate that cer- 
tain airline accidents might not have occurred, had automatic al- 
titude deviation callouts been made (cf. 6 and 7). If a reliable 
algorithm to detect unnoticed altitude deviations during the ap- 
proach could be developed, this could be a valuable component of 
a SYNCALL system. One algorithm for this is proposed for con- 
sideration in the section entitled "Refinement of a synthesized 
voice approach callout system". 

• Consistent with a SYNCALL system that calls out deviations 
only would be the incorporation of the Ground Proximity Warning 
System (GPWS) modes into SYNCALL. SYNCALL as tested included 
warnings for excessive sink rate and for deviation below the 
glideslope. The "excessive rate of closure with terrain" mode of 
the GPWS logically could be added to the set of conditions SYN- 
CALL monitors and calls out. 

If this reasoning is correct, a SYNCALL system that automati- 
cally called out deviations in airspeed, sink rate, glideslope, 
localizer, altitude, and terrain closure, only when deviations 
actually existed, could be a valuable addition to airline cockpit 
s y s t ems • 
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CONCLUSION 


The concept of an automatic approach callout system using 
synthesized speech definitely deserves further study. Improve- 
ments in flight performance by airline pilots were obtained for 
approaches that have high manual and visual workload for the pi- 
lots. By extrapolation, similar improvements might be obtained 
for any high workload approach. For the One-Engine Approach, for 
which SYNCALL made inappropriate deviation callouts, airspeed 
performance was worse with SYNCALL than with PNF. Reliability of 
normal altitude approach callouts was comparable for PNF on the 
line and in the simulator and for SYNCALL in the simulator. How- 
ever, SYNCALL was more reliable than PNF for making deviation ap- 
proach callouts in the simulator. Pilots generally favored the 
concept of SYNCALL and judged it more reliable than PNF callouts. 
They suggested modifications before it would be appropriate for 
operational use. It was concluded that SYNCALL improved flight 
performance for non-precision approaches, that a SYNCALL system 
should make deviation callouts only. For consistency, it is 
recommended that the modes of the Ground Proximity Warning System 
(GPWS) be incorporated into a SYNCALL system. The detrimental 
effects on performance associated with inappropriate deviation 
callouts led to the further conclusion that such callouts should 
be designed out of the system. Finally, in the section entitled 
"Refinement of a synthesized voice approach callout system", the 
results and conclusions of the experiment are used to develop 
suggestions for improvements to SYNCALL before further testing. 
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REFINEMENT OF A SYNTHESIZED VOICE APPROACH CALLOUT SYSTEM 


Clearly the SYNCALL system as configured for this experiment 
is not appropriate for air transport use. This is evidenced by 
the fact that some of the data for some of the measures of system 
effectiveness found SYNCALL to be no better than or, in some 
cases, not as good as PNF callouts. For example, SYNCALL was 
judged as more reliable than PNF callouts; but it was also judged 
less informative, less helpful, and less coordinating than PNF 
callouts. Tnese pilot rating scale data agreed with the callout 
system performance data on reliability of the callouts being 
made. And, the rating data on the scales informative/ distract- 
ing, helpful/ detrimental, and coordinating/ disruptive were at 
least indirectly supported by the data on percent overlap of SYN- 
CALL callouts with other voice communications in the cockpit. 
Finally, the long list of pilot recommended modifications and the 
"highly desirable" and "desirable" ratings they assigned to these 
modifications argue against implementation of the system exactly 
as configured for the experiment, 

Wnat, then, might an appropriate SYNC ALL system be like? It 
is difficult to derive a coherent recommendation for system 
design from only one type of data. However, when the flight per- 
formance data is considered together with the pilot rating data, 
the callout reliability data, and the percent overlap data, a 
consistent argument can be made in support of a general design 
for a synthesized voice callout system with certain variables 
still undefined or requiring experimental comparison among alter- 
natives, Such a system, once configured, would then have to un- 
dergo testing using a methodology similar to that used in this 

study, 

SYNCA LL System Components 

While SYNCALL was designed as a single experimental system so 
as to test several different types of synthesized approach cal- 
louts, it was composed of different functional parts. These are 
listed below: 

1) normal altitude callouts e.g. 1000 AFL, 500, 400 ... 

2) normal position callouts e.g, outer marker, G/S inter- 

cept, ... 

3) deviation callouts e.g, airspeed +15, left of localizer, 

• » , 

4) critical altitude callouts, e.g, decision height, MDA 

5) priority logic 

6) callout timing and repetition logic 

7) tolerance values 

The experimental design tested the entire SYNCALL system com- 
pared to the entire PNF system of callouts. Thus no conclusions 
can be drawn from the results regarding the relative contribution 
of different functional components of each callout system to the 
observed flight performance, system performance, and pilot rating 
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differences. However, analysis of the data was done in a way 
that permits rational speculation on the contribution of indivi- 
dual SYNC ALL components to the observed differences. And the 
resulting speculations can be used to derive further hypotheses 
for testing . 

When looking for the underlying reasons for the observed 
differences associated with the two systems, one naturally looks 
for differences in performance between corresponding components 
of the two callout systems. Eacn of the seven system components 
listed above will be treated in turn. 

1 ) normal al titude callouts Both the PNF and SYNCALL made the 
altitude callouts reliably. Although reliability of PNF altitude 
callouts was slightly less (7M£) than that of SYNCALL altitude 
callouts (35%) , there were no approaches for either system for 
which no altitude callouts were made. Neither system was 100% 
reliable in making altitude callouts since both the PNF and SYN- 
CALL gave priority to calling deviations when faced with a demand 
for simultaneous altitude and deviation callouts. Furthermore, 
the reliability of normal altitude callouts made on the line 
(33%) was nearly equal to that of SYNCALL (35%) when the line 
callout reliability was adjusted to account for the high percen- 
tage of visual approaches on the line. Since both systems 
resulted in high reliability for altitude callouts, it is diffi- 
cult to attribute any flight performance differences to any 
differences in the altitude callout component of SYNCALL versus 
PNF. Rather than introduce automatic normal altitude callouts 
when no particular benefit to flight performance would be expect- 
ed, altitude callouts by SYNCALL if implemented at all ought to 
be designed as a back-up - i.e, SYNCALL should make them only if 
the PNF is unable to make them or forgets to make them for some 
reason. It must be noted, however, that the pilots actually 
favored altitude callouts by SYNCALL more than they did SYNCALL 
deviation callouts. The discussion of deviation callouts, cal- 
lout timing and repetition logic, and tolerance values below may 
shed some light on this apparent discrepancy. 

2) normal position callout s SYNCALL was more reliable than 
PNF for these callouts, 

3) deviation cal louts SYNCALL was more reliable than PNF in 
calling out deviations in airspeed, sink rate, localizer posi- 
tion, and glideslope position: SYNCALL 90 % , PNF 49%, where in- 
stances of deviations requiring callouts were counted rather than 
number of callouts since a given deviation could conceivably 
result in multiple callouts. This difference between the perfor- 
mance of the two callout systems, then, suggests that the devia- 
tion callout component of SYNCALL was important to the observed 
improvement in flight performance with SYNCALL. 

*0 -Critic al aJjLi tjid_e callouts A direct comparison of the two 
callout systems for this component is not possible because these 
callouts are required only if the PF has not announced that the 
runway is in sight by the time the critical altitude (DH or MDA) 
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is reached, SYNCALL could not detect whether this was the case, 
so it always called the critical altitudes. Thus a, comparison 
between PNF and SYNCALL for critical callout reliability is not 
meaningful . 

5) priority logic SYNCALL was designed with different priori- 
ties assigned to different types of callouts to cover situations 
when more than one callout was triggered. The criteria used for 
this priority assignment were 1) Make the most time-critical cal- 
lout first, and 2) Make only less time-critical callouts after- 
wards only if they still apply. 

Decision height and MDA were judged prior to the experiment 
to be the most t ime- cr i t ica 1 of all the callouts since the 
pilot-flying must act immediately on the information conveyed by 
these callouts so as not to compromise the safety of the approach 
by descending below rainimuras when this is not warranted. 

The next highest priority was assigned to deviation callouts; 
and the deviation parameters were ordered within this from higher 
to lower priority as glideslope, indicated airspeed, sink rate, 
and local izer , 

Lowest priority was given to normal altitude callouts and 
normal position callouts. Since these by definition were not ex- 
pected to occur simultaneously, no priorities within these were 
assigned. In fact, on rare occasions localizer intercept, 
glideslope intercept, and/or outer marker passage occur simul- 
taneously , 

This priority logic had an effect on the reliabilty of dif- 
ferent types of callouts made by SYNCALL. The most reliable were 
the D H and MDA callouts (100/6), Next in reliability were devia- 
tion callouts (90%) . Lowest in reliability for SYNCALL were al- 
titude callouts (83£). The callouts that SYNCALL made most reli- 
ably were the ones that were given the highest priority when the 
system was designed. It is noteworthy that even SYNCALL, which 
presumably d etected 100? of all the callouts that should have 
been made, did not have time to speak all the callouts in cases 
where critical altitude and deviation callouts were required. 

We have only anecdotal evidence regarding the priority system 
used by the PNF to decide which of several simultaneously re- 
quired callouts to make. Very likely it is much more complex 
than the unid imensional one employed by the SYNCALL system. It 
probably takes into account a wide range of real world contextual 
factors such as the type of approach being made, the type of ter- 
rain under the approach path, the weather, the wind conditions 
(above and beyond considerations made in computing the approach 
speed), whether or not the runway is in sight, and the amount of 
excursion beyond tolerance when deciding between multiple devia- 
tion callouts. Also, the PNF knows that deviations in certain 
parameters are associated with deviations in other parameters and 
that correcting one may effectively correct the other. Excessive 
airspeed and excessive sink rate exemplify this type of associa- 
tion, If one is present, the other may also be present, espe- 
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daily on non-precision approaches. Whether the PNF calls 
airspeed or sink rate first will depend, then, on many factors. 
We cannot hope to duplicate the human's priority system so SYN- 
CALL will function as the PNF does because we do not have all the 
information, properly weighted, that the PNF brings to the situa- 
tion , 

However, it was possible to observe, both on the flight deck 
and in the simulator, some general patterns of PNF behavior when 
confronted with the requirement simultaneous callouts. By noting 
what callouts the PNF made in these situations and when he made 
them, we can infer something about his priority system. 

Tne PNF employed a different timing strategy than SYNCALL for 
integrating altitude callouts and deviation callouts. Often a 
PNF deviation callout was made immediately after a 100-foot cal- 
lout, presumably to ensure that the 100-foot callout was made on 
time. SYNCALL, in contrast, called out a deviation as soon as it 
was detected and postponed any 100-foot callouts that were re- 
quired during ennunciation of the deviation callout. Further, 
such an altitude callout was skipped entirely if the postponement 
would nave caused it to be spoken more than 50 feet late. Had 
SYNCALL employed the strategy used by the PNF, it might have been 
more reliable for normal altitude callouts with no detriment to 
deviation callout reliability, Tnis would have been at the ex- 
pense of SYNCALL's deviation callout timeliness, however. 

6) callout t imin g ajLd repetition l ogic Callout timeliness is 
closely linked to callout timing and repetition logic. The PNF 
strategy seems to be to wait for 100-foot altitude intervals and 
to call out any deviations at these times, just after making a 
100-foot callout. SYNCALL, on the other hand, called a deviation 
as soon as it detected one and checked again in 5 s. Assuming a 
700 FPM rate of descent, the 5-s interval would be the same as 
checking every 58 feet or about twice the rate of the PNF in 
terras of altitude loss. For a 1200 FPM rate of descent, which 
would trigger a SYNCALL sink rate deviation callout, the altitude 
loss between SYNCALL repetitions would be 100 feet, the same as 
for PNF callouts. Thus, only when sink rate itself was exces- 
sively high was the repetition rate by SYNCALL equivalent to that 
of PNF, Otherwise, SYNCALL made deviation callout repetitions 
for glideslope and airspeed twice as often as the PNF would have 
done. The approximately double repetition rate of SYNCALL devia- 
tion callouts compared to PNF callouts is a second difference 
between the two systems that could have caused the observed 
flight performance difference in airspeed and sink rate for the 
non-precision approaches. Whether increased deviation callout 
reliability or increased callout repetiton rate or some combina- 
tion of these produced the improvement in flight performance with 
SYNCALL cannot be determined from the data since the two factors 
are confounded. The pilots did judge the system to have an unac- 
ceptably high rate of nuisance deviation callouts. If a syn- 
thesized voice approach callout system were to be designed for 
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further testing, serious consideration should be given to ways of 
reducing deviation callout repetition rates to a level that would 
not be judged a nuisance by the pilots. Nuisance callouts could 
also be reduced by changing the decision logic for triggering re- 
petitions of a callout for the same deviation. Pilots commented 
that if they saw that the pilot-flying was in the process of 
correcting ,a deviation, they did not repeat it even if the actual 
value was still outside tolerance. Only if they observed no 
change in deviation or an increased deviation did they say they 
would call it again. The SYNCALL localizer and glideslope devia- 
tion callout repetition logic was in fact designed in exactly 
this way. The logic for the airspeed and sink rate deviation 
callouts was not. That is, SYNCALL made a repetition of the de- 
viation after 5 s if the value was still outside tolerance re- 
gardless of whether the pilot was in the process of correcting or 
not. In addition to changing the rate of deviation callout re- 
petitions, or perhaps instead of changing this rate, a greater 
reduction in nuisance callouts might be achieved by making the 
logic for airspeed and sink rate deviation callouts be the same 
as that used for the localizer and glideslope deviation callouts 
and that which the pilots report they use: repeat the deviation 
callout only if no correction is being made. This approach to 
reducing nuisance callouts would also be consistent with the 
finding that pilots were satisfied with the repetition rates for 
each type of deviation callout and yet wanted a reduction in nui- 
sance callouts. 

The timeliness of SYNCALL’s initial deviation callouts, how- 
ever, could have been one of the major contributing factors to 
the improved flight performance with SYNCALL in that deviations 
un-noticed by the Pilot-Flying were brought to his attention im- 
mediately rather than at the next 100-foot interval of descent. 
Again, the data do not permit direct inference of this conclu- 
sion, but serious consideration should be given to preserving 
this aspect of SYNCALL if future systems are designed for test- 
ing , 

7) tolerance values SYNCALL was designed with tolerance 
values that were 25% to 33% greater than the "book" values used 
by the PNF. This was done in hopes of ensuring that the PNF 
* would continue to watch for and call out observed deviations even 
when SYNCALL was in use. Instead, the effect of SYNCALL on PNF 
deviation callout reliability was to reduce it from 49% to 24J. 
This suggests a potential problem, that of the PNF relying too 
heavily on SYNCALL to monitor for and make deviation callouts. 
Perhaps still larger differences between SYNCALL and PNF "book" 
tolerances would have relegated SYNCALL to more of a back-up 
role. One would want to see PNF deviation callout reliability at 
least as good with SYNCALL as without SYNCALL in order to be cer- 
tain that SYNCALL was not creating a new problem by its introduc- 
tion. According to the pilot judgment data, the SYNCALL toler- 
ances, particularly for airspeed, may indeed have been too tight. 
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A future system might be designed with the 
suggested by the pilots: +10 KTS and -5 KTS 


airspeed tolerances 


Summary of Recommended Changes to S Y N CAL_L 

On the basis of the analysis of the possible contributions of the 
different components of SYNCALL to the observed improvements in 
flight performance and in callout reliability, and to the ratings 
and suggested modifications by the pilots, a tentative SYNCALL 
system for further testing can be proposed. The system would 
make deviation callouts in airspeed, glideslope, localizer, and 
sink rate using the same tolerances for all but airspeed. The 
airspeed tolerance would be changed to +10 KTS and -5 KTS from V 
( APPR ) • This would give the following tolerances: 


GLIDESLOPE 
AIRSPEED 
SINK RATE 

2000 -1000 AFL 
1000 - 300 AFL 
BELOW 300 AFL 
LOCALIZER 


+/- 1 1/3 DOTS 
V(APPR) +10,-5 KTS 

2500 FPM 
1200 FPM 
1000 FPM 
+/- 1/2 DOT 


In addition, consideration should be given to including a 
callout and associated tolerance for VOR course tracking and for 
Back Course Localizer course tracking. One could use the values 
suggested by one of the participating pilots as starting points. 
These were 1 1/2 dots for a Localizer Back Course and 2 dots for 
a VOR radial as final approach course. 

It will be remembered that for the experiment the functions 
of the GPWS were incorporated into the SYNCALL system on a non- 
duplicating basis. Any future SYNCALL system ought to include 
the excessive terrain closure warning of the GPWS for consisten- 
cy, While not used in this study, the "Sink on Take-off" func- 
tional warning of the GPWS would presumably also be incorporated 
into a modified SYNCALL system. The other GPWS modes, which all 
are designed to pertain to sink rate and glideslope in various 
aircraft configurations would be unnecessary because the informa- 
tion they are designed to convey and some additional information 
as well, e.g. actual sink rate values, aircraft above glideslope, 
is given by SYNCALL. In support of this merging of GPWS with 
SYNCALL there is the semantic differential data which indicates 
that the pilots “ound SYNCALL to be more informative, more help- 
ful, more dependable, and more coordinating that either the GPWS 
Whoop Whoop Pull Up or the GPWS Glideslope message. Also, one of 
the 31 modifications suggested by the pilots was "Eliminate GPWS 
and Replace with SYNCALL". This item received a mean rating of 
2,15 +/- 0.90 and a median response of 2 (desirable). 

Merging SYNCALL and GPWS warning functions would provide the pi- 
lot with more information, and in particular would state the 
problem and the degree of the problem. It can be suggested but 
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not tested after the fact that such a system in place of the GPWS 
system could have alerted the crew of a 727 that accidentally 
landed short due to unintentional excessive sink rate on an over- 
water approach despite the triggering of a Whoop Whoop Pull Up 
message from the GPWS. ( 6) 9 

The repetition logic for the proposed SYNC ALL system should 
have the feature used by the PNF and by the experimental SYNCALL 
system for glideslope and localizer deviation callouts; once a 
deviation has been called, it should not be called again unless 
the deviation is not being corrected. The time interval of 5 s 
before a recheck of airspeed, glideslope, or sink and 10 s before 
a recheck of localizer deviation would be expected to work well 
provided this change of logic is implemented. 

The priority assignment of the different deviations was, 
from highest to lowest, GPWS Terrain Closure, Glideslope, 
Airspeed, Sink Rate, and Localizer, Some of the pilots and two 
of the instructor pilots suggested that Sink Rate deviations 
should be given higher priority than Airspeed deviations. They 
thought this would be especially useful for the non-precision ap- 
proaches, for which glide path information is not available. An 
experimental comparison of these two priority assignments will 
have to be made using precision and non-precision approaches and 
measuring flight performance, callout reliability, callout timel- 
iness, and pilot ratings before one can be chosen over the other. 
Such an experiment should include more extreme conditions of 
wind, gusts, and wind direction and velocity changes (wind sheer) 
than was possible in this study. 

Once a particular type of deviation had been called, the next 
highest priority parameter should be checked before the first is 
checked again. Without this provision, a SYNCALL system could 
conceivably call one type of deviation while ignoring others un- 
til that deviation was corrected. 

The proposed SYNCALL system would not make normal altitude 
callouts, PNF callout reliability both in the experiment and on 
the line was, for all practical purposes, equal to that of SYN- 
CALL for normal altitude callouts. Looked at the other way, SYN- 
CALL was unable to improve on the performance of the pilot-not- 
flying for normal altitude callout reliability; in part this was 


9, The GPWS in this accident was of the type that was designed 
to call out "Whoop Whoop, Pull Up" for any of four different 
problems: excessive terrain closure, excessive sink rate, sinking 
on take-off, and flaps or landing gear not in landing configura- 
tion. Newer models of this system have been designed to state 
the actual problem that triggered the warning, partially conform- 
ing to the suggestion here to merge GPWS and SYNCALL design 
features. One can also hypothesize that for the accident refer- 
enced above, the use of this other type of GPWS might have 
prevented the accident. 
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because SYNC ALL placed higher priority than did the PNF on cal- 
ling out deviations as soon as detected, often to the detriment 
of SYNCALL normal callout reliability. There exist, however, 
rare instances when normal altitude callouts by SYNCALL could ac- 
tually perform the warning function served by the deviation cal- 
louts. If for any reason, the crew is not aware of descent to 
low altitude and the PNF for that reason does not make the normal 
altitude callouts, SYNCALL could alert them to this by calling 
out altitudes only on these rare occasions. This could be 
designed into SYNCALL by having it automatically armed at the 
start of each approach to make warning altitude callouts as fol- 
lows: 

Callout Wording AFL Altitude 


BELOW 1000 FEET 
BELOW 500 FEET 
BELOW 100 FEET 


900 feet 
400 feet 
30 feet 


These warning altitude callouts would be made by SYNCALL only if 
an abnormal descent in terms of sink rate was occurring. Abnor- 
mal sink rate would be defined as either excessively high ox ex- 
cessively low sink rate. Appropriate values cannot be given 
here. As a starting point, one could use sink rate in excess of 
SYNCALL sink rate deviation tolerances or sink rate less than 100 
F PM to cover unintentional gradual descents. It must be emphati- 
cally stressed that this part of the proposed SYNCALL system is 
extremely tentative and is derived from general human factors 
design principles to avoid false alarms and not from the data 
collected and analyzed in this study. A far better way of ob- 
taining sink rate values to use as activators for warning alti- 
tude callouts by a SYNCALL system would be to analyze sink rate 
histories from a large sample of different types of approaches 
that were terminated successfully and compare the resulting means 
and standard deviations to sink rate histories for approaches 
that resulted in controlled flight into terrain (CFIT), Unless a 
combination of values could be found that would virtually elim- 
inate false alarms while virtually guaranteeing that all in- 
stances of unintent ional descent would be called out, such a 
feature should not be incorporated into an automatic approach 
callout system such as SYNCALL, 

The Ten-Foot normal altitude callouts deserve special men- 
tion, however. Unlike the rest of the normal altitude callouts, 
SYNCALL was definitely more reliable than PNF for these callouts 
(100/S compared to 77X). Tnis was noted independently during and 
after the simulator session by some of the participating pilots. 
In some case, pilots said they felt they were late making ten- 
foot callouts and that the other pilot made the ten-foot callouts 
late. Also, captains noted that despite a procedural requirement 
that they make ten-foot callouts as pi 1 o t-no t - f 1 y i ng , that they 
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as pilot in command felt obliged to look outside the cockpit dur- 
ing the flare and touchdown. 10 Interestingly, there were no 
differences in ten-foot callout reliability as a function of po- 
sition flown. Rather, both captains and first officers simply 
were not as reliable as SYNCALL in making these callouts. Assum- 
ing that the timing of SYNCALL ten-foot callouts could be made 
acceptable, such callouts could be an effective means of normal 
altitude information transfer at a point in the approach where 
SYNCALL would not be making deviation callouts. The timing of 
SYNCALL ten-foot callouts might be improved an algorithm utiliz- 
ing sink rate to remove the inherent delay in the altitude cal- 
lout . 

For purposes of design consistency and simplicity from the 
viewpoint of the user, SYNCALL should not make normal position 
callouts. Even though SYNCALL was more reliable than PNF for 
this type of callout, inclusion of such callouts in a SYNCALL 
system would possibly dilute the warning function of the system 
in that pilots would then not associate SYNCALL callouts strictly 
with warnings. Also, SYNCALL was not able to make normal posi- 
tion callouts with 100? reliability. This might tend to lessen 
pilot confidence in the system. 

The question of critical altitude callouts (Decision Height, 
MDA, 100 above MDA ) is difficult. On the one hand, these are 
warnings only if the pilot-flying does not have the runway in 
sight. If the pilot-flying has announced the runway in sight, 
then the decision height or MDA callout and Missed Approach Point 
(MAP) callout will not be needed. The latter would also be ex- 
tremely difficult to design into a SYNCALL system unless there 
were access to Inertial Navigation system or RNAV system data. 
On the other hand, SYNCALL was much more reliable than PNF for 
these callouts (100? compared to 58?, 48? and 52? respectively 
for 100 above MDA, MDA, and Decision Height callouts by PNF), In 
those instances when the decision height callout serves its func- 
tion and the pilot-not- flying (for monitored approaches) or 
pilot-flying (for standard approaches) initiates a go-around, it 
is critical that the callout be made in a timely and reliable 
fashion. Were a SYNCALL system to make decision height and MDA 
callouts in a more timely way than the PNF is able to do, it 
might be useful to have SYNCALL make such callouts for all ap- 
proaches with the provision that approach procedures be modified 
to have the pilot-not-flying call out "Going Around" if SYNCALL 
says Decision Height and the Pilot-Flying has not announced the 
runway in sight. In the case of MDA, possibly giving the task of 


10. Sometime after completion of the experimental approaches 
for SYNCALL and following informal presentation of these comments 
to the flight training department of the participating airline, 
the airline's procedures were modified to permit the captain as 
pilot-not-flying at his discretion to omit the ten-foot callouts. 
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calling out MDA to SYNCALL could make the task of monitoring for 
the MAP easier for the PNF. This entire discussion of Critical 
altitude callouts by SYNCALL and possible changes to approach 
procedures is presented here to stimulate further thought on the 
question and not to advise for or against implementation of crit- 
ical altitude callouts in any future SYNCALL systems for testing. 

Summary of Proposed Modifications .to SYNCALL 

To summarize, it is suggested on the basis of the findings 
from this study that SYNCALL be re-configured essentially as an 
approach warning callout system and be merged with the GPWS as 
one entire system. For the system to be effective, the recommen- 
dations to eliminate nuisance alarms will have to be followed. 
Also the repetition logic will have to be changed. Finally, on 
the basis of the detrimental effects observed due to inappropri- 
ate airspeed deviation callouts during the single-engine ap- 
proach, no inappropriate deviation or other warning callouts can 
be tolerated. If these cannot be designed out of the system, or 
at best designed with a pi 1 o t- sel ec tabl e deactivation switch, 
then they should not be included as part of it at all. The 
resulting system will then have to be tested in a flight simula- 
tion study similar in comprehensiveness of systems and approach 
environment to that used for this study. 
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ALTITUDE AND APPROACH CALLOUTS BY SPEECH SYNTHESIZER 

Carol. A. Simpson, NRC Associate 
NASA Ames Research Center 

In preparation for a flight simulation study of synthesized speech 
Approach Callouts in an American Airlines DC-10 flight training simulator, 
phoneme codes for the ML-1 Speech Synthesizer 1 were programmed to de- 
velop a Vocabulary of words and short phrases that could be appropriate-^ 
ly concantenated to produce all of the required callouts including 
readout of actual altitude, airspeed, and rate of sink. Callout 
messages composed from this Vocabulary were then tested for intelli- 
gibility. 

In order to maximize the chances for misunderstanding of any messages 
that might be of low intelligibility, the testing conditions were 
chosen so as to be more difficult than any of the conditions that were 
expected to actually occur in the flight simulator. Background 
noise shaped to simulate DC-10 cockpit noise was presented at a 
level 10 db above the level of the loudest speech segment in the 
Approach Callout messages. The airline pilots who attempted to 
understand the callouts were given no prior exposure to the callouts 
and were told only that the messages they would hear were similar 
to the callouts made during the approach by the p i lot -not -f lying . 

The pilots represented several different airlines and thus were 
not necessarily familiar with the particular set of callouts or 
the phraseology used in this study. The results are presented in 
outline form on. page 3 . The mean articulation score of 9 0.4$ correct 
under these extremely difficult testing conditions indicates that 
this set of callout messages is very intelligible and suggests that 
these messages would be acceptable for use in the flight simulator. 


Manufactured by Vocal Interface Division, Federal Screw Works, Troy, 
Michigan 
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As a further test of the individual words used to compose altitude callouts, 
a separate study was conducted with messages consisting entirely 
of altitudes ranging from Flight Level HkO to 10 feet radio altimeter. 

Two pronunciations of the digits for the altitude callouts were tested, 
a "phonetic" pronunciation and a "normal" pronunciation. In addition, 
pilots were asked to give their preferences for type of pronunciation 
and also for altitude phraseology, e.g. one three thousand versus 
thirteen thousand for 13000. These results are presented on pages 
k and 5. The high articulation scores of 9^.7 to 100$ correct 
at a s/n of -10 db also suggest that these individual words are highly t 
intelligible and adequate for use in the simulator. No significant 
difference in intelligibility was obtained for the two types of pronunciation. 
A majority of the pilots, however, preferred the "normal" pronunciation 
over the "phonetic" pronunciation, except for the digit 9, which a 
majority of pilots preferred to have pronounced as "niner". Therefore, 
it is recommended that "normal" pronunciation be used for all digits 
except "niner". For altitude phraseology, nearly all pilots tested 
preferred the standard ATC phraseology, e.g. one three thousand for 
13000 and one three hundred for 1300. It is therefore recommended that 
the synthesizer follow ATC phraseology rules in making altitude 
callouts . 

Finally, pilot preferences for speech rate, voice pitch, and signal-to- 
noise ratio for this synthesizer were obtained. These results are presented 
on page 6 • In general, pilots preferred a voice pitch in the 
middle of the male voice pitch range with no significant differences 
in voice pitch or speech rate as a function of presence or absence 
of cockpit noise. The mean preferred signal- to-noise ratio was db. 

It should be noted that the standard deviation of these preferences 
was quite large, 7 .8 db. In order to accomodate those pilots requiring 
a high signal-to-noise ratio, :it is recommended that the callouts 
be presented at a s.ignal-to-noise ratio of 8 db, slightly less than 
the mean + 1 standard deviation. 
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Approach Callouts at s/N = -10db: Intelligibility in DC-10 cockpit noise 


Stimuli: Standard American Airlines DC-10 Approach Callouts pre- 

sented over headphones with simulated DC-10 cockpit noise 
as background; s/N = -10 db. Each Callout presented 
once only. No prior familiarization with the messages. 


Subjects: 7 Airline pilots 


Results: 


Percent Word Articulation for 27 callouts Containing a 
total of 97 words 


Pilot Number 

Age 

Word Articulation = Percent Correct 

1 

31 

90.7 

2 

34 

95.9 

3 

*6 

88.7 

4 

33 

100.0 

5 

39 

89.7 

6 

37 

88.7 

7 

42 

79.4 

Mean 

37 

90.4 
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Phonetic versus Normal Pronunciation: Intelligibility Comparison 


Stimuli: Altitude readouts ranging from 10 feet to Flight Level 44o 

presented over headphones with simulated DC-10 cockpit noise 
as background; s/N = -lOdb. 


Subjects: .5 airline pilots 


Test Material: The "Phonetic" and "Normal" sets contained identical 

altitudes and differed only in the pronunciation of the digits 
3, 4, and 5. The digit 9 was pronounced "niner" in both sets. 


Digit 

3 

4 

5 


Results : 


Phonetic Pronunciation 
tree LtriJ 
fower [fowc'J 
fife [fajfj 
niner [ najn?J 


Normal Pronunciation 
three [QriJ 
four I'iTu'J 
five ffajvU 
nine fnajn^J 


Percent Word Articulation for 2 sets of 2. 4 altitudes containing 

a total of 171 words 


Pilot Number 

"Phoneti c 

Pronunciation" 

"Normal 

Pronunciation" 


Set 1 

Set 2 

Set 1 

Set 2 

1 

9b. G 

100.0 

100.0 

97.7 

2 

9b.o 

100.0 

100.0 

100.0 

3 

100.0 

100.0 

100.0 

100.0 

4 

83.1 

100.0 

100.0 

100.0 

5 

100.0 

100.0 

98.8 

100.0 

Mean 

9^.7 

100.0 

99.8 

99.5 


Analysis: There was no significant difference in aritculation scores 

for the two types of pronunciation. 


* All data were obtained using a VOTRAX ML-1 voice synthesizer, manufactured 
by Vocal Interface Division, Federal Screw Works, Troy, Michigan 
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'Airline Pilot Preferences for Pronunciation of Altitude Readouts 


Subjects: 12 airline pilots 


Results : 


Digit 

Phonetic Pronunciation 

Normal Pronunciation 

Either One OK 

3 

1 

10 

1 

k 

3 

8 

1 

5 

3 

9 

0 

9 

8 

2 

2 


Altitude 

ATC Phraseology* 

Common Phraseology* 

Either One OK 

13000 

11 

1 

0 

1300 

12 

0 

0 


Numbers are number of pilots responding in each category 


Conclusion: A majority of airline pilots prefer normal pronunciation for 

the digits 3, and 5 and phonetic pronunciation of the 
digit 9 as niner . They prefer the standard ATC phraseology 
for altitudes over the Common phraseology, e.g. one three 
thousand over thirteen thousand and one thousand three hundred 
over thirteen hundred. 


+ ATC Phraseology = one three thousand and one thousand three hundred 
Normal Phraseology =« thirteen thousand and thirteen hundred ~~ 







Airline Pilot Preferences for speech rate, voice pitch, and signal- 
to-noise ratio for the Votrax ML-l Speech Synthesizer 


Stimuli: Synthesized Altitude and Approach Callout Messages presented 

over headphones with and without simulated DC-10 cockpit noise as 
background. 


Suujects: Commercial airline pilots 


Procedure: For speech rate and voice pitch preferences, each pilot 

adjusted the appropriate knobs (continuous analog adjustment) first 
with no noise, then with background noise while listening to the 
Callout messages repeated 10 times each. For signal-to-noise 
preferences, pii.ots told experimentor to adjust the. message level 
up or down until satisfied with the level of the messages relative to 
the noise level. Experimentor made adjustments in 10 , 5, and 1 db 
units using a calibrated attenuator. 

Results : 


No Noise 


Mean 

Stand. Dev. 

N 

Knob Setting 

Measured Freq. 

Kb St. 

mm 

Hate 

5.7 


1 1.1 


13 

Pitch 

5.3 

91 Hz 

± 1.0 

4-9 Hz 
-8 Hz 

13 

With DC-10 

Cockpit 

Noise 

Rate 

5.6 


± 1.3 


13 

Pitch 

5.6 

92 Hz 

± 0.5 

+7 Hz 
-3 Hz 

13 

s/n 

+3.h db 


± 7.6 db 

1 

9 


* Note: The relationship between units marked on the pitch control 

knob of the VOTRAX and the resulting fundamental frequency of the 
synthesized voice is nonlinear. 

































VOCABULARY OF WORDS AND PHRASES USED TO GENERATE APPROACH CALLOUTS 


Item Number 

1 . 

2 . 

3. 

4. 

5. 

6 . 

T. 

8 . 

9- 

10 . 

11 . 

12 . 

13. 

lit. 

15. 

16. 

IT. 

18. 

19- 

20 . 

21 . 

22 . 

23. 

24. 

25. 

26. 

2.7 • 

28. 

29. 

30. 


Vocabuary Item 

zero. 

one 

two 

three. 

four 

five 

six 

seven 

eight 

niner 

ten 

eleven 

twelve 

thirteen 

fourteen 

fifteen 

sixteen 

seventeen 

eighteen 

nineteen 

hundred 

thousand 

twenty 

thirty 

forty 

fifty 

s :i xty 

seventy 

eighty 

ninety 
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VOCABULARY ITEMS (cent.) 


Item Number 

Voorabulary Item 

31. 

Flight Level 

32. 

M3L 

33. 

Above Field 

3*N 

feet 

35. 

minus 

36. 

plus 

37. 

/message-initial pause/ 

38. 

/phrase or message-final pause/ 

39. 

Rate of climb is 

ijo. 

You're above the glideslope. 

hi. 

You're belov/ the glideslope. 

h2. 

You are right of the localizer. 

h3. 

You are left of the localizer. 

hh. 

and 

h5' 

Whoop Whoopi Pull Up I 

h6. 

Passing through 18000 . 

hr. 

Leaving 11000. 

h8. 

Glideslope intercept. 

h9. 

Localizer intercept. 

50. 

Outer Marker 

51. 

Decision Height 

52. 

100 feet above MDA. 

53. 

Sink is 

5h. 

MDA 

55- 

Airspeed 
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APPENDIX B 


Flight Scenario for One of Four Experimental Orders 


Captain: A. Takeoff 

B. ATC Departure 

C. Holding and/or Steep Turns and/or Stalls 
(Instructor Pilot’s discretion) 

D. Vectors for Approach 

1. (SYNCALL) Backcourse 5R (10 KTS cross-wind) 

with complete missed approach. 

Vectors or reset to: 

2. (SYNCALL) ILS DUAL LAND approach and landing 

(10 KTS cross-wind) 

E. Takeoff Engine Failure between VI and V2 (no wind) 

V A t* rv K* <3 AK 1 f A • 

3. (SYNCALL) Two-engine ILS to 100 feet (10 KTS 

. quartering wind) and missed approach. 
Vectors or reset to: 

F. Two-engine landing. 

F/0 : G. Takeoff 

H. ATC Departure 

I. Holding and/or Steep Turns and/or Stalls 
(Instructor Pilot's discretion) 

J. Vectors for Approach 

1. (SYNCALL) Backcourse 6R (10 KTS crosswind) 

with complete missed approach. 

Vectors or reset to: 

2. (SYNCALL) Coupled ILS APPCH ONLY and land 

(no wind) 

K. Takeoff engine Failure between VI and V2 (no wind) 

Vectors or reset to: 

3. (SYNCALL) Two-Engine ILS to 200 feet and land 

(10 KTS quatering wind) 
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Captain : 


F/0: 


Captain : 


L. Takeoff and vectoring for approaches 

4. (PLTNFLY ) VOR to 7L/R using Raw Data with 

rejected landing at 10 feet. (10 KTS 
crosswind) Vectors or reset to: 

5. (PLTNFLY) ILS DUAL LAND to SINGL LAND to go- 

around (10 KTS wind) Vectors or reset 
to : 

4. (PLTNFLY) VOR to L/R using Raw Data with 

rejected landing at 10 feet. (10 KTS 
crosswind) Vectors or reset to: 

5. (PLTNFLY) ILS DUAL LAND to SINGL LAND to 

APPCH ONLY and go-around(10 KTS wind) 

M. Engine Failure or shutdown, then: 

N. Engine Fire, the Vectors or reset to: 

6. (PLTNFLY) One-Engine Landing (VFR and no wind) 


Notes: SYNCALL s Synthesized Voice Approach Callout System 

PLTNFLY = Pilot Not Flying Callout System 
The above scenario shows the assigment of pilot-flying 
and callout system used to Presentation Order 1. 
Presentation Orders 2, 3, and 4 were derived by altering 
the pilot to fly first (Captain or First Officer) and 
the callout system to be used first (SYNCALL or PLTNFLY) 
The order of the approaches flown: (1, 2, 3, 4, 5, 6) 
was constant for all Presentation Orders in accordance 
with American Airlines' recurrent training syllabus. 


-3 2- 



PILOT EVALUATION OF APPROACH CALLOUT SYSTEMS 


The speech synthesizer was programmed to make all of the altitude and 
deviation approach callouts. This was done to give you a chance to 
experience the range 6f capability of such a system. In your own pro- 
fessional judgment, certain types of callouts, either by the speech 
synthesizer or by the pilot-not-flying, may be more or less desireable. 

The next two pages contain "preference grids". The different types of 
callouts are listed down the side - first for SYNCALL, the speech 
synthesizer, then (on the next page) for the PILOT-NOT-FLYING. Also 
4 different types of approach conditions are listed across the top 
of each preference grid: 

1) Night VFR 2) Abnormal or 3) Day VFR 4) IFR to 

Emergency Mini mums 

Starting with the grid for SYNCALL, put a check in the appropriate box 
for each type of callout that you think would be useful and desireable 
for the type of approach listed. Do this for each type of approach 
condition. Then repeat the process for the PILOT-NOT-FLYING grid. 

Check as many or as few callouts for each type of approach condition 
and system as you want. If you want no callouts for a particular, type 
of approach and system, check the lowest box, "none". 



SYNCALL 



Night VFR 

j Abnormal /j Day VFR 
j Smergencyj 

IFR to 
minimums 

ieesese^^s 

IHHI 



Localizer Intercept i 

i 

I 


OM and Altitude AFL 1 

; ... . . 

< 

... . . | 


1000 Feet AFL j 

i 


100 Feet Above MDA j 


t 


MDA j 

i 

» 

j 

. i j 

Decision Height ! I 

... . . 1 ! 

i 

I 


500 Feet, Airspeed & • j 

Sink Rate j J 

i 

* 

j 


Hundred root CaL Louts 

i ! 

t 


Ten Foot Callouts 

J « 

i ! 

i L 


G/S Deviations 


i 

! 


LOC Deviations 





Sink Rate Deviations 





Airspeed Deviations 





Hone (no caL Louts) 
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PILOT-NOT-FLYING 



Night VFR 

I Abnormal/ 
j Emergency 

Day VFR 

IFR to i 
min imams 

Glideslope Intercept 

j 

i 

1 

1 


Localizer Intercept j 




OM and Altitude AFL 





1000 Feet AFL 





100 Feet Above MDA 





MDA j 




Decision Height 






500 Feet, Airspeed & 
Sink Rate 

! 



Hundred "root Ca Ll outs 





Ten Foot Callouts 

i 



G/S Deviations 





LOC Deviations 





Sink Rate Deviations 





Airspeed Deviations 





None (no caL Louts) 
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Assume for the moment that the PILOT-NOT-FLYING callouts were modified 
to be exactly the way you specified In the preference grid. Rate your 
system on the following scale by placing an "X" in the appropriate space. 


unsafe 


• • • « 

• _ » • ^ 

no highly 

preference desireable 


Now assume for the moment that SYNCALL were implemented exactly the way 
you specified in the preference grid. Rate your system on the scaTe“Eelow, 


unsafe 


• . • _ • 

no highly 

preference desireable 


Now assume tjie PILOT-NOT-FLYING callouts remain the same as they are now. 
Rate the current system of pi lot-not-f lying callouts on the scale below. 


» 

• 

• 

• • 

• • 

• 

M 9 

• 

• 

unsafe 


no 


highly 



preference 


desireable 


Now assume, that the SYNCALL system were Implemented exactly as you experienced 
it today in the simulator. Rate this system on the scale below. 


unsale 


• » • . t 

’ no highly 

preference desireable 


On the next four scales below, rate the Intelligibility of the four types 
of speech which you just heard in the simulator. 

SYNCALL 

t * « * » i 

jjj * • * ggj * * ’“'lOO* 




LA TRACON 



1 ( 50 ? 


( 5 ? 


LAX TOWER 



Totf? 


PILOT-NOT-FLYING 


If a SYNCALL system were installed in the cockpit, it might be desireable 
to build In some limits on the conditions for the different callouts. 
Listed below are several technologically feasible constraints for such 
a system. Using the following scale, rate each suggestion by placing a 
1, 2, 3, 4, or 5 in front of it. 


1 Highly desireable 

2 Desireable 

3 No preference 

4 Undesireable 

5 Unsafe 


Deviation callouts not activated until 500 feet above field and 
lower. 

Hundred Foot callouts activated only for coupled approaches. 

Hundred Foot callouts can be deactivated by Captain at Captain's 
discretion for VFR approaches. 

Glldeslope deviations can be deactivated at Captain's discretion 
for VFR approaches. 


(In the blank spaces, add any limits that you would want to impose 
on such a system and rate your suggestions with the same scale. 



PILOT RECOMMENDATIONS FOR MODIFICATION OF SYNCALL 


Introduction 

The tolerance windows for the SYNCALL deviation callouts were picked as "ball 
park" values so as to test the general concept of an automatic approach callout 
system. I figured the values might need changing once a group of line pilots 
had actually flown SYNCALL in the simulator. When you were flying SYNCALL, you 
may have thought the deviation callouts were made too frequently or maybe not 
often enough and that the allowances on airspeed, sink rate, glideslope, and 
localizer were either too loose or too tight to work well in the real world. 

I would like to get your recommendations for setting the tolerances for each 
type of deviation callout. And I would like to also get your recommendations 
for the priority assignments to each type of callout. Please answer each of 
the following questions based on your short experience with SYNCALL and on your 
extensive experience flying the line. And feel free to add comments ! 

If you have any questions about what I'm asking, please call me (collect) at 
NASA. When you call, just say that you were in the SYNCALL study. When you 
have finished filling this out you can use the stamped, self-addressed envelope 
to mail it back to me. 


Return to: 

Carol A. Simpson, PhD 
Mail Stop 239-2 
NASA Ames Research Center 
Moffett Field, CA 94025 

Call collect: 415/965-6128 



1. SYNCALL tolerance "windows 


The chart below shows the tolerance “windows" that were set for SYNCALL when you flew 
it in the simulator. In each case, the "window" is wider than the tolerances you have 
in your operating manual for the pi lot-not- flying and the flight engineer to use when 
making deviation callouts. 


AIRSPEED Approach (“Bug") Speed 

+ 7 KTS 

SINK RATE Between 2000 and 1000 AFL 

2500 fpm 

Between 1000 and 300 AFL 

1200 fpm 

Below 300 AFL 

1000 fpm 

LOCALIZER 

1/2 dot 

GLIDESLOPE 

1 and 1/3 dot 


In the chart below please fill in the values that you_ think SYNCALL should use. Note 
that for the airspeed you don't have to choose the same amount on the negative side 
as on the positive side. For example, you could choose -20 and +3. Keep in mind that 
you want to avoid nuisance callouts but that you want the system to tell you about any 
thing really dangerous. 

If you would not want SYNCALL to make a particular type of deviation callout at all, 
regardless of the size of the "window", please put a check in the box at the far right 
of the chart on the line corresponding to that type of callout. Note, for example, 
that you might want SYNCALL on sink rate for only some but not all the altitudes listed 


✓ IF DON'T WANT 

TYPE OF CALLOUT “WINDOW" THIS TYPE CALLOUT 


AIRSPEED Approach ("Bug") Speed 



SINK RATE Between 2000 and 1000 AFL 



Between 1000 and 300 AFL 



Below 300 AFL 



LOCALIZER 



GLIDESLOPE 


















2. Deviation callout repetition rates and "active" segments 

The chartbelow shows the rates at which the different SYNCALL deviation callouts were 
repeated. For example, airspeed deviations were repeated once every 5 seconds as long 
as the aircraft remained outside the "window". The chart also shows the point in the 
approach where each callout became "active". At that point SYNCALL would start checking 
for that type of deviation. Before that SYNCALL would not make any callouts even If 
the aircraft was outside the "window". For example, SYNCALL started checking for sink 
rate deviations at 600 feet AFL. Finally, the chart shows the point where SYNCALL was 
automatically "turned off" for each type of deviation so as to avoid distractions at 
low altitude. For example, SYNCALL stops checking for glideslope deviations at 100 feet 
AFL. 


TYPE OF CALLOUT REPETITION RATE START CHECKING STOP CHECKING 


AIRSPEED 

5 sec. 

600 AFL 

100 AFL 

SINK RATE 

5 sec. 

600 AFL 

100 AFL 

LOCALIZER 

10 sec. 

at LOC CAP 

100 AFL 

GLIDESLOPE 

5 sec. 

at G/S CAP 

100 AFL 


In the chart below, please fill in the values that you think SYNCALL should use for 
repetition rate, start checking, and stop checking. As on page 1, if you would not 
want SYNCALL to make a particular type of deviation callout at all, regardless of the 
repetition rate or the "active" segment, please put a check in the box at the far 
right side of the chart on the line corresponding to that type of callout. 


, IF DON'T WANT 

TYPE OF CALLOUT REPETITION RATE START CHECKING STOP CHECKING THIS TYPE CALLOUT 


AIRSPEED 





SINK RATE 





LOCALIZER 





GLIDESLOPE 




























3, SYNCALL Priority System 


SYNCALL systematically missed certain callouts for certain approaches due to its priority 
system and the timing of the callouts. For example, on a VOR 7R Approach, MDA was 436 AFL. 
So SYNCALL called “100 feet above MDA" at 536 AFL and did not make the 500 AFL callout. 

If a deviation callout coincided with an altitude callout, SYNCALL made the deviation 
callout and skipped the altitude callout. The chart on the left lists the types of SYNCALL 
callouts, including the GPWS Whoop Whoop for Terrain (Mode 2), in their order of priority 
from highest to lowest. If you would want a different order of priorities, please list 
the types of callouts in the order you would want in the chart in the middle. If there 
are any of these callouts that you would not want SYNCALL to make at all, please list them 
in the chart to the right. If, on the other hand, you like the priorities the way they 
are, please check the box below the three charts. 

SYNCALL should not 


Current Priorities 

Your Recommended Priorities 

make these callouts 

DH or MDA 





GPWS Mode 2 (Terrain) 





Glideslope Dev. 





Airspeed Dev. 





Sink Rate Dev. 





Localizer Dev. 





Altitude Callouts 



1 



□ Leave Priorities as they are 
4. Suggested Modifications to SYNCALL 

Listed below are all the suggestions given by one or more pilots who flew SYNCALL. Please 
consider each one and give It a rating using the scale at the top of the list. Put your 
rating for each suggestion on the line just to the left of that suggestion. For example, 
If you find a particular suggestion "highly desirable", put a "1" on the line next to the 
suggestion. If you find it "undesirable", then put a "4" beside it. Do this for each 
suggestion. After you finish, go back and pick out any- and all of the suggestions that 
for you would be essential in order for you to be willing to accept SYNCALL - i.e. if 
SYNCALL were not modi f i ed acccrdi ng to these particular suggestions, then you would not 
want it in your airplane. 

Note that the list continues on the next page. 


RATING SCALE 


Unsafe 

Undesirable 

No Preference 
or Can't Decide 

Desirable 

Highly Desirable 

5 

4 

3 

2 

1 


. Put your 

V here if Rating here 

Essential (1,2, 3, 4, 5) SUGGESTED MODIFICATIONS TO SYNCALL 


□ 


Have ON/OFF switch in case of erroneous info or distraction from SYNCALL 


-D 4- 



ODD 0 DDDDDDDDDOODDDDDD 


Suggested Modifications to SYNCALL continued - 

RATING SCALE 


unsafe undesirable no preference desirable highly desirable 

or can't decide 

5 4 3 2 I 

_ Put your 

v here if rating here 

essential (1,2, 3,4, 5) SUGGESTED MODIFICATIONS TO SYNCALL 

Provide Test switch for accuracy check 
Minimize frequency of deviation callouts 
Delay callouts during radio transmissions (If possible) 

Eliminate GPWS & replace with SYNCALL 
Add a warning in case airspeed gets below Alpha Speed 
Shorten message wording 
Speed up SYNCALL' s speech rate 
Eliminate nuisance callouts 

Add other warnings, e.g. engine fire, loss of cabin pressure, etc. 

Add immediate action checklist Items 
Provide manual volume control 
Increase volume 

Increase tolerance on localizer deviations 
Increase tolerance on glideslope deviations 
Increase tolerance on airspeed deviations 

Spell out SYNCALL and pi lot-not-f lying responsibilities more clearly 

At the 50 ft callout, call out airspeed and sink. Then eliminate 
the 40, 30, 20, 10 callouts by SYNCALL 

Inhibit airspeed and sink rate deviation callouts during abnormal or 
emergency approaches 

Automatically change SYNCALL parameters depending on type of approach 

Call Localizer Capture, then Localizer Intercept 

Read out Flight Guidance Mode Annunciators after LAND mode has 
been armed 

(continued on next page) 
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Suggested Modifications to SYNCALL continued - 


unsafe undesirable 
5 4 


RATING SCALE 

no preference desirable highly desirable 

or can't decide 

3 2 1 


'/here if 
essential 

□ 

n 

□ 

a 

□ 

□ 

□ 

□ 

a 


Put your 
rating here 
(1,2, 3,4, 5) 

Call out "Below MDA" If aircraft descends below MDA (unless pilot 
has pressed a "disable" switch) 

On non-precision approaches, give sink rate deviations top priority 

Obtain FAA & Company agreement not to use CVR recordings of SYNCALL 
against the crew 

Have SYNCALL call the flaps setting at 1000 AFL 

Have SYNCALL say "Check Flaps" at 1000 AFL 

Make voice quality for deviations more alerting compared to altitude calls 

Have SYNCALL make deviation callouts only (f.e. only If something's wrong) 

Have SYNCALL make altitude callouts only (leave deviations to pllot-not- 
f lying and flight engineer) 

If aircraft balloons on landing, have SYNCALL repeat 10-foot callouts 
as appropriate, e.g. "50, 40, 30, 20, 30, 20, 10“ 


5. Background Data 

Please fill In the chart below 0 £]y for the type aircraft you were cu rrent In at the 
time that you flew SYNCALL 
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6. Preference Grids for desired callouts for different types of Approach Conditions 

The two preference grids below are like the ones you filled out after flying SYNCALL In the simulator. Please fill 
them out using the same Instructions as before. That Is, put a check In the appropriate box for each type of callout 
that you think would be useful and desirable for the type of approach listed. Do this for each of the four types of 
approach conditions. By leaving a box blank, you are saying you don't want that type of callout for that type of 
approach or that you are not sure^that you would want It. Check the box only If you really would want SYNCALL to make 
that callout for that type of approach. 

There are two preference grids. Fill out the. one .on .the .left assuming SYNCALL stayed the way It was when you flew It 
In the simulator. Then fill out the one on the right assuming SYNCALL was changed according to your own recomnendatlons 
on the preceedlng pages of this evaluation. 


i 

0 

1 


USE'miS GRID FOR SYNCALL THE HAY IT MAS HHEN YOU FLEW IT IM THE SIHUUTOR 


Typt of Approach Conditions 


Typt of Callout 

Night VTR 

Abnormal/ 

Baargancy 

Day VFR 

IFR to 
alnlmsss 

Qlldaslopa Intareapt 





Local Lzar intareapt 



1 


OK and AltLtuda AFL 





1000 Ptac AFL 





100 Face Abova MDA 





HDA. 





Doc !• Lon Haight 





300 rant, Alrapand & 
Sink Rata 





HtlRWAa root cliiduf* 





Tan Foot Callouts 





O/S Darla t Ions 





LOC Darla t Ions 





Sink Rata DavLatlons 





Alrspaad Davis t Ions 





Wna (no callouts) 






C5T THIS GRID FOR SYNCALL THE HAY IT HOUID BE IF IT HERE CHANGED THE HAY YOU KAYE 
RECOffCNDCD IN THIS EVALUATION 

Type of Approach Conditions 


Type of Callout 

Night VFR 

Abnormal/ 

Baargancy 

Day yrt 

IFR to 
minimum* 

Qlldaslopa In tare apt 





Local ixar In tare apt 





OK and Altltuda AFL 





1000 Faat AFL 





100 faat Abova HDA 





KDA- 





Dae Is Lon Haight 





300 faat, Alrspaad & 
Sink Rata 




| 





■ 

Tan Foot Callouts 




■HUB 

0/3 Dariations 





LOC DavLatlons 





Sink Rata DavLatlons 




■ ■ 

Alrspaad Davlations 




■ ■ 

Wma (no callouts) 




MB 


That's the end. Thank you very much, and I'll let you know how this all turns out. 
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